next_offset pagination in auth logs

[ClovisIRex]

Hi, I'm trying to pull auth logs from the last 30 days using:

admin_api.get_authentication_log(api_version=2, kwargs=params)

I did manage to get them but I can't manage to paginate through the results via the 'next_offset' param. I always get the same next_offset in the response, I think I may use the wrong format or something but it isn’t well documented enough in the examples.

I printed the debug data to the console, for example in here I try to send 3 requests- first request has next_offset set 0, second request has an updated next_offset from the first request response but the third requests uses the same next_offset- it did not get a different value in the response.

If you can please provide me with Postman collections or code examples with pagination through the results(we have over 20,000 results over the last month) it would be appreciated
Thanks in advance

[DuoKristina]

The paging param name is offset, not next_offset. https://duo.com/docs/adminapi#response-paging

I see the param name is given incorrectly in the auth logs section of that same doc page and will get that fixed.

ETA - this was wrong; disregard

[ClovisIRex]

I tried renaming it to offset but I get no response now...

[DuoKristina]

Hm, OK. Investigating what it should actually be (since the other endpoints take offset in the gets)...

[DuoKristina]

OK, next_offset is correct for authlogs. Looking more closely at your screenshot you might not include all the needed info.

You need to pass in both of those values as `next_offset`.

The offset at which to start record retrieval. This value is provided in the metadata in the form of a 13 character date string in milliseconds and the event txid. Both of these values must be provided when used, separated by a comma.

Looks like this is only sending the txid?

[ClovisIRex]

I did pass them both at first and got same results. tried passing only the id and still no difference. I now tried using offset while sending and next_offset while receiving. still no use

[DuoKristina]

hmm

kristina@api examples % python -m duo_client.client --ikey $IKEY --skey $SKEY --host $HOST --method GET --path /admin/v2/logs/authentication mintime=1712015808000 maxtime=1713383808000 limit=1                                                              
200 OK
{
    "response": {
        "authlogs": [
            {
                "access_device": {
                    **auth event details snipped**
                "txid": "0b7d377b-887a-4b7a-96b8-3017e97ec2ef",
                "user": {
                    **auth event details snipped**"
                }
            }
        ],
        "metadata": {
            "next_offset": [
                "1713383798454",
                "0b7d377b-887a-4b7a-96b8-3017e97ec2ef"
            ],
            "total_objects": 2
        }
    },
    "stat": "OK"
}
kristina@api examples % python -m duo_client.client --ikey $IKEY --skey $SKEY --host $HOST --method GET --path /admin/v2/logs/authentication mintime=1712015808000 maxtime=1713383808000 limit=1 next_offset=1713383798454,0b7d377b-887a-4b7a-96b8-3017e97ec2ef
200 OK
{
    "response": {
        "authlogs": [
            {
                "access_device": { **auth event details snipped**
                "txid": "e691974c-b062-4e88-a1d2-3fef5bfd7f91",
                "user": {
                    **auth event details snipped**
                }
            }
        ],
        "metadata": {
            "next_offset": [
                "1713383776499",
                "e691974c-b062-4e88-a1d2-3fef5bfd7f91"
            ],
            "total_objects": 2
        }
    },
    "stat": "OK"
}

Limited to return one of two events; passing in the next_offset info from the first response returns the next event (different txid and resulting next_offset info).

[ClovisIRex]

thanks for the help but it still doesn't work for me..
I tried changing the next_offset from a list to a string separated by comma but to no avail.

I dont get a different next_offset, I get the same results over and over, but only the last result has a different offset, strange:

should I wait between requests or something?
also, I only get 100 results at a time instead of the 1000 that is says on the limit.

[ClovisIRex]

Im using this function admin_api.get_authentication_log as follows:

auth_logs = admin_api.get_authentication_log(api_version=2, kwargs=params)
next_offset_from_api = ','.join(auth_logs.get("metadata").get("next_offset"))
logger.info("[DUO] Next offset from response: %s",next_offset_from_api)
params['next_offset'] = next_offset_from_api

should I use a different function?

I saw on the client that it has json_paging_api_call

[ClovisIRex]

Also, I couldnt run your command separately, if you could please send me a cURL it would be great

[ClovisIRex]

I think i'll rephrase my issue. I want to pull all auth logs data from the last x days, this data will be later parsed and plotted.

is there a simpler way to do that with the client?

[darsh-ambaliya]

Facing a similar issue. I am having trouble paginating the results in the activity logs, telephony logs, and authentication logs. Could someone please provide me with an example of the proper way to receive and paginate these logs in Python code?

[ClovisIRex]

Hi any update on this?

[jay-law]

Probably related - #286