<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic Re: Data Recording From REST API Get Recording is not consistent in PAM Self-Hosted Discussions</title>
    <link>https://live.paloaltonetworks.com/t5/pam-self-hosted-discussions/data-recording-from-rest-api-get-recording-is-not-consistent/m-p/1305406#M7881</link>
    <description>&lt;P&gt;well that is definitely weird, the same call should produce the same result set&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Not sure what that python library is doing with datetime conversions - is it possible that the timestamps are being translated into different localisation? In US locale 3/1/2025 is March 1st, but in most European local it is 3rd January..? Probably not relevant if you are testing from the same machine each time.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Or perhaps a pagination issue - your code doesn't seem to have any loop to handle retrieving the next page of data so it must be the calling function that is responsible for handling this. Perhaps the function that calls recordings() is not looping correctly to handle pagination - limit and offset are the important values here. We start with offset=0 and a set page size such as limit=100. If the first call returns 100 records (ie the value of limit) then set offset=100 and call the API again. Keep repeating adding 100 (limit) to offset each time, until the number of records returned is less than 100 (limit). Then you know you have retrieved all the records. If you aren't properly setting limit and offset in the function that calls recordings() then the results could be a bit random.&lt;/P&gt;</description>
    <pubDate>Mon, 19 May 2025 08:40:12 GMT</pubDate>
    <dc:creator>Jon_Musker</dc:creator>
    <dc:date>2025-05-19T08:40:12Z</dc:date>
    <item>
      <title>Data Recording From REST API Get Recording is not consistent</title>
      <link>https://live.paloaltonetworks.com/t5/pam-self-hosted-discussions/data-recording-from-rest-api-get-recording-is-not-consistent/m-p/1305403#M7878</link>
      <description>&lt;P&gt;Dear All,&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;I retrieve data of recording via REST API (Get Recording).&lt;/P&gt;&lt;P&gt;First tried compare to Second tried are different.&lt;/P&gt;&lt;P&gt;&lt;B&gt;The result of First tried:&lt;/B&gt;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper" image-alt="Field_Content_versions_oct20__068Vy00000Wnu0FIAR.png"&gt;&lt;img src="https://live.paloaltonetworks.com/t5/image/serverpage/image-id/92333i8F007FBD2195B32C/image-size/large?v=v2&amp;amp;px=999" role="button" title="Field_Content_versions_oct20__068Vy00000Wnu0FIAR.png" alt="Field_Content_versions_oct20__068Vy00000Wnu0FIAR.png" /&gt;&lt;/span&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;B&gt;The result of Second tried :&lt;/B&gt;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper" image-alt="Field_Content_versions_oct20__068Vy00000Wn7KnIAJ.png"&gt;&lt;img src="https://live.paloaltonetworks.com/t5/image/serverpage/image-id/92331i9784707F2CFB4D92/image-size/large?v=v2&amp;amp;px=999" role="button" title="Field_Content_versions_oct20__068Vy00000Wn7KnIAJ.png" alt="Field_Content_versions_oct20__068Vy00000Wn7KnIAJ.png" /&gt;&lt;/span&gt;If we check on the result of second tried, recording with start &lt;B&gt;03/01/2025 06:08:00 AM&lt;/B&gt; and &lt;B&gt;03/01/2025 06:11:23 AM&lt;/B&gt; (Yellow Background) are Not Exist in the result of First tried.&lt;/P&gt;&lt;P&gt;I have checked the parameter of MaxRecords on PSM (via PVWA) is 8000 and i use data retrieve limitation 1000 in my source code, but actually the amount of data that be retrieved only under 200 data.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Additional information : Source code to retrieve data from Cyberark REST API using Pyhton. Cyberark version is 12.6.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Please any advice. Thanks&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
      <pubDate>Tue, 13 May 2025 13:44:08 GMT</pubDate>
      <guid>https://live.paloaltonetworks.com/t5/pam-self-hosted-discussions/data-recording-from-rest-api-get-recording-is-not-consistent/m-p/1305403#M7878</guid>
      <dc:creator>CyberArkAlumni</dc:creator>
      <dc:date>2025-05-13T13:44:08Z</dc:date>
    </item>
    <item>
      <title>Re: Data Recording From REST API Get Recording is not consistent</title>
      <link>https://live.paloaltonetworks.com/t5/pam-self-hosted-discussions/data-recording-from-rest-api-get-recording-is-not-consistent/m-p/1305404#M7879</link>
      <description>&lt;P&gt;Odd. I notice that the two missing recordings are very long sessions - started on 1 March 6am and ended on 2 March 3am - around 21 hours each. &lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;My first question; is it possible that you ran the first query before those two sessions ended? I guess not since that was two months ago, but perhaps we can just confirm it.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Secondly perhaps there was a delay in uploading the recording from PSM into the Vault due to the unusual size of the recordings? so that the recording may have finished on 2 March but the PSM did not successfully upload to the Vault until you were running the query.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;What parameters were passed to the API call?&lt;/P&gt;</description>
      <pubDate>Wed, 14 May 2025 09:36:35 GMT</pubDate>
      <guid>https://live.paloaltonetworks.com/t5/pam-self-hosted-discussions/data-recording-from-rest-api-get-recording-is-not-consistent/m-p/1305404#M7879</guid>
      <dc:creator>Jon_Musker</dc:creator>
      <dc:date>2025-05-14T09:36:35Z</dc:date>
    </item>
    <item>
      <title>Re: Data Recording From REST API Get Recording is not consistent</title>
      <link>https://live.paloaltonetworks.com/t5/pam-self-hosted-discussions/data-recording-from-rest-api-get-recording-is-not-consistent/m-p/1305405#M7880</link>
      <description>&lt;P&gt;Dear @Jon Musker​&amp;nbsp;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Yes. the remote session is in march, 1 2025. we have confirm to user that they did monitoring. So that why the remote session is 75310 second (20 Hour 10 Minute). And the report is generate on April 2025.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Actually there are another several different data between First tried and Second tried.&lt;/P&gt;&lt;P&gt;&lt;B&gt;Result of First Tried :&lt;/B&gt;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper" image-alt="Field_Content_versions_oct20__068Vy00000X6bMrIAJ.png"&gt;&lt;img src="https://live.paloaltonetworks.com/t5/image/serverpage/image-id/92365i375EF38863FD1B0B/image-size/large?v=v2&amp;amp;px=999" role="button" title="Field_Content_versions_oct20__068Vy00000X6bMrIAJ.png" alt="Field_Content_versions_oct20__068Vy00000X6bMrIAJ.png" /&gt;&lt;/span&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;B&gt;Result of Second Tried :&lt;/B&gt;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper" image-alt="Field_Content_versions_oct20__068Vy00000X6LEwIAN.png"&gt;&lt;img src="https://live.paloaltonetworks.com/t5/image/serverpage/image-id/92363i2A18BECBB42026F9/image-size/large?v=v2&amp;amp;px=999" role="button" title="Field_Content_versions_oct20__068Vy00000X6LEwIAN.png" alt="Field_Content_versions_oct20__068Vy00000X6LEwIAN.png" /&gt;&lt;/span&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;We can see above. The data that not show in first tried (yellow background) have remote session duration only under 9000 second.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;B&gt;This is my code to retrieve recording data via REST API (using python):&lt;/B&gt;&lt;/P&gt;&lt;P&gt;def recordings(url, token, timeStampStartDate, timeStampEndDate, limit, offset, sort):&lt;/P&gt;&lt;P&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;requests.packages.urllib3.disable_warnings(InsecureRequestWarning)&lt;/P&gt;&lt;P&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;headers = {'Authorization': token, 'Content-Type': 'application/json'}&lt;/P&gt;&lt;P&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;params = {"FromTime": timeStampStartDate, "ToTime": timeStampEndDate,&lt;/P&gt;&lt;P&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;"limit": limit, "offset": offset, "sort": sort}&lt;/P&gt;&lt;P&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;response = requests.get(url, params=params, headers=headers, verify=False)&lt;/P&gt;&lt;P&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;statusCode = response.status_code&lt;/P&gt;&lt;P&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;if statusCode == 200:&lt;/P&gt;&lt;P&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;responseApi = response.json()&lt;/P&gt;&lt;P&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;return ([statusCode, responseApi])&lt;/P&gt;&lt;P&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;else:&lt;/P&gt;&lt;P&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;return ([statusCode, response])&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;=====================================================&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;This how I call the function to retrieve:&lt;/P&gt;&lt;P&gt;responseApi = recordings(url,token,int(start_dt.timestamp()),int(until_dt.timestamp()),limit,offset,sort)&amp;nbsp;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
      <pubDate>Thu, 15 May 2025 03:53:18 GMT</pubDate>
      <guid>https://live.paloaltonetworks.com/t5/pam-self-hosted-discussions/data-recording-from-rest-api-get-recording-is-not-consistent/m-p/1305405#M7880</guid>
      <dc:creator>CyberArkAlumni</dc:creator>
      <dc:date>2025-05-15T03:53:18Z</dc:date>
    </item>
    <item>
      <title>Re: Data Recording From REST API Get Recording is not consistent</title>
      <link>https://live.paloaltonetworks.com/t5/pam-self-hosted-discussions/data-recording-from-rest-api-get-recording-is-not-consistent/m-p/1305406#M7881</link>
      <description>&lt;P&gt;well that is definitely weird, the same call should produce the same result set&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Not sure what that python library is doing with datetime conversions - is it possible that the timestamps are being translated into different localisation? In US locale 3/1/2025 is March 1st, but in most European local it is 3rd January..? Probably not relevant if you are testing from the same machine each time.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Or perhaps a pagination issue - your code doesn't seem to have any loop to handle retrieving the next page of data so it must be the calling function that is responsible for handling this. Perhaps the function that calls recordings() is not looping correctly to handle pagination - limit and offset are the important values here. We start with offset=0 and a set page size such as limit=100. If the first call returns 100 records (ie the value of limit) then set offset=100 and call the API again. Keep repeating adding 100 (limit) to offset each time, until the number of records returned is less than 100 (limit). Then you know you have retrieved all the records. If you aren't properly setting limit and offset in the function that calls recordings() then the results could be a bit random.&lt;/P&gt;</description>
      <pubDate>Mon, 19 May 2025 08:40:12 GMT</pubDate>
      <guid>https://live.paloaltonetworks.com/t5/pam-self-hosted-discussions/data-recording-from-rest-api-get-recording-is-not-consistent/m-p/1305406#M7881</guid>
      <dc:creator>Jon_Musker</dc:creator>
      <dc:date>2025-05-19T08:40:12Z</dc:date>
    </item>
    <item>
      <title>Re: Data Recording From REST API Get Recording is not consistent</title>
      <link>https://live.paloaltonetworks.com/t5/pam-self-hosted-discussions/data-recording-from-rest-api-get-recording-is-not-consistent/m-p/1305407#M7882</link>
      <description>&lt;P&gt;Dear @Jon Musker​&amp;nbsp;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;first of all thanks for response.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;The information that I can share are limit = 1000 , if we set time range 1 month (30 days) it will loop 30 time send API request. &lt;/P&gt;&lt;P&gt;So looping is based on 1 day. And the data in 1 day less than 1000 (around 200 to 500 data). &lt;/P&gt;&lt;P&gt;Why I loop per day, because to avoid MaxRecords parameter if the data that will be retrieve is more that MaxRecords.&lt;/P&gt;&lt;P&gt; &lt;/P&gt;&lt;P&gt;I have tried  the source code to Dev Environment many time. And the source code  run smoothly. The data in dev environment only less than 200.&lt;/P&gt;&lt;P&gt;But when I run to production environment, the problem happen. The data in production environment in 1 month around 10.000 data.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Does Cyberark take data from the database or from the log file when generating JSON?&lt;/P&gt;&lt;P&gt;Looks like there is a certain process that lock the data that make it is locked and can not be retrieved. &lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
      <pubDate>Mon, 19 May 2025 13:32:26 GMT</pubDate>
      <guid>https://live.paloaltonetworks.com/t5/pam-self-hosted-discussions/data-recording-from-rest-api-get-recording-is-not-consistent/m-p/1305407#M7882</guid>
      <dc:creator>CyberArkAlumni</dc:creator>
      <dc:date>2025-05-19T13:32:26Z</dc:date>
    </item>
    <item>
      <title>Re: Data Recording From REST API Get Recording is not consistent</title>
      <link>https://live.paloaltonetworks.com/t5/pam-self-hosted-discussions/data-recording-from-rest-api-get-recording-is-not-consistent/m-p/1305408#M7883</link>
      <description>&lt;P&gt;try removing the sort. If you are just retrieving all records then the order you retrieve them will be largely irrelevant; I guess it is possible that it is confusing the issue in between the calls, since each api call is stateless - it's not as if the system has a frozen dataset that you are iterating over. After you have retrieved all the records to the app, sorting them in memory will be trivial.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;I don't think you should assume that each day will have less than 1000 records; limit/offset should be setup to work reliably and then just request the actual date range (ie one month). Breaking it down arbitrarily into one day at a time seems like you don't trust the API, but you must trust your code otherwise it isn't ready to release! Once you have it correct it should work properly whether the limit is 5 or 5000, the only reason to vary limit would be for performance. Also when searching by date range in any complex database there is always a risk of edge cases near the start and end of the range (what time zone, what counts as start or end time etc). I particularly note that you say you are reading data in one day at a time, but at least two of the 'weird' records are recordings that span two different days.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Make sure you have enough error handling; maybe add some trace logging to your app to see what is happening on each loop (how many records returned, what value for limit/offset is used on each pass)&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Data should be coming from the database rather than logs afaik, it involves joining multiple tables to get all the session data.&lt;/P&gt;</description>
      <pubDate>Mon, 19 May 2025 15:18:21 GMT</pubDate>
      <guid>https://live.paloaltonetworks.com/t5/pam-self-hosted-discussions/data-recording-from-rest-api-get-recording-is-not-consistent/m-p/1305408#M7883</guid>
      <dc:creator>Jon_Musker</dc:creator>
      <dc:date>2025-05-19T15:18:21Z</dc:date>
    </item>
    <item>
      <title>Re: Data Recording From REST API Get Recording is not consistent</title>
      <link>https://live.paloaltonetworks.com/t5/pam-self-hosted-discussions/data-recording-from-rest-api-get-recording-is-not-consistent/m-p/1305409#M7884</link>
      <description>&lt;P&gt;Hello @Krida Kusuma​&amp;nbsp;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Have you received an answer to your question? If so, we kindly encourage you to share it with us.&lt;/P&gt;&lt;P&gt;Your insights could be valuable to other community members in the future.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Thank you, Isar.&lt;/P&gt;</description>
      <pubDate>Thu, 22 May 2025 07:21:24 GMT</pubDate>
      <guid>https://live.paloaltonetworks.com/t5/pam-self-hosted-discussions/data-recording-from-rest-api-get-recording-is-not-consistent/m-p/1305409#M7884</guid>
      <dc:creator>CyberArkAlumni</dc:creator>
      <dc:date>2025-05-22T07:21:24Z</dc:date>
    </item>
  </channel>
</rss>

