429 - Too Many Requests.... after a request every 20 seconds?

Hello, all.

Been battling this API for 2 weeks now. We’re finally pulling data – our department head wants reporting on authentications, but doesn’t care for the UI, so I’ve been tasked with developing a data-pull from the /admin/v2/logs/authentication endpoint. I got it to pull the last 24 hours, and we’re able to insert it all into a custom DB on or side to source the reporting. Shiny.

Next request: can you pull everything from this year? Sure… shouldn’t be too hard. Except I’m getting a 429 -Too Many Requests within the first couple pulls (limit 1000, mintime and maxtime set 12 hours apart.) I started sleeping the program for various intervals, and I’m finding that if I set the interval for any less than 30 seconds, I start getting 429’s. One run – set at 20 seconds – failed after the third request. 3 requests in 40 seconds, and it died.

This is insane. I’m the only one using this ikey/skey. We use Duo for MFA across the company (about 900-950 authentications per business day since the COVID shutdown, maybe 100-200 daily before that). But in terms of the API, I’m it. Just me and my Visual Studio. But it’s been too much, apparently, for the API to handle, so I’ve been crawling along, pulling 45 days worth of data over the last 90 minutes.

Is this application of the API somehow unexpected? How could I possibly be taxing the server at this rate?

Hey there, we recommend limiting Admin API request on the logs endpoint to one per minute. More requests than that may cause the results you’ve seen. We’ll note this in the Admin API documentation as well.