Skip to content

[cisco_duo] Set request rate limits - #11441

Merged
chrisberkhout merged 3 commits into
elastic:mainfrom
chrisberkhout:cisco-duo-rate-limit
Oct 17, 2024
Merged

[cisco_duo] Set request rate limits#11441
chrisberkhout merged 3 commits into
elastic:mainfrom
chrisberkhout:cisco-duo-rate-limit

Conversation

@chrisberkhout

@chrisberkhout chrisberkhout commented Oct 16, 2024

Copy link
Copy Markdown
Contributor

Proposed commit message

[cisco_duo] Set request rate limits

The Duo Admin API has rate limiting. It doesn't return rate limit
headers, but it does enforce limits with HTTP 429 responses.

For some endpoints, the API documentation specifies "a rate limit of 50
calls per minute". This as also been observed on the authentication
logs endpoint.

This changes sets a limit of 0.5 calls/second or 30 calls per minute
for all data streams and inputs.

HTTP 429 responses continue to be treated as errors.

API documentation: https://duo.com/docs/adminapi

I considered returning { "events": [], "want_more": false } when a 429 response is received, however, with the new settings we expect to never exceed the limit, and if the limit is repeatedly exceeded, treating it as an error will help with troubleshooting.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices
@chrisberkhout chrisberkhout added Integration:cisco_duo Cisco Duo bugfix Pull request that fixes a bug issue Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Oct 16, 2024
@chrisberkhout chrisberkhout self-assigned this Oct 16, 2024
@chrisberkhout
chrisberkhout requested a review from a team as a code owner October 16, 2024 17:46
@elasticmachine

Copy link
Copy Markdown

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

Comment thread packages/cisco_duo/changelog.yml Outdated

@efd6 efd6 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be worthwhile assessing for a 429 in the CEL code and returning the state unmodified, but with an empty events array.

Co-authored-by: Dan Kortschak <dan.kortschak@elastic.co>
@chrisberkhout

Copy link
Copy Markdown
Contributor Author

It may be worthwhile assessing for a 429 in the CEL code and returning the state unmodified, but with an empty events array.

That was my original plan, but I changed my mind. If it happens occasionally that would be good because it would just wait and try again after the interval. But we're only going to use 60% of the expected allowance, so if we do get 429s, especially a lot of them, it'll be good to see them as errors.

@chrisberkhout
chrisberkhout enabled auto-merge (squash) October 17, 2024 07:10
@chrisberkhout
chrisberkhout merged commit fd15e94 into elastic:main Oct 17, 2024
@elasticmachine

Copy link
Copy Markdown

💚 Build Succeeded

History

cc @chrisberkhout

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package cisco_duo - 2.0.3 containing this change is available at https://epr.elastic.co/search?package=cisco_duo

harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 4, 2025
The Duo Admin API has rate limiting. It doesn't return rate limit
headers, but it does enforce limits with HTTP 429 responses.

For some endpoints, the API documentation specifies "a rate limit of 50
calls per minute". This as also been observed on the authentication
logs endpoint.

This changes sets a limit of 0.5 calls/second or 30 calls per minute
for all data streams and inputs.

HTTP 429 responses continue to be treated as errors.

API documentation: https://duo.com/docs/adminapi

---------

Co-authored-by: Dan Kortschak <dan.kortschak@elastic.co>
harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 5, 2025
The Duo Admin API has rate limiting. It doesn't return rate limit
headers, but it does enforce limits with HTTP 429 responses.

For some endpoints, the API documentation specifies "a rate limit of 50
calls per minute". This as also been observed on the authentication
logs endpoint.

This changes sets a limit of 0.5 calls/second or 30 calls per minute
for all data streams and inputs.

HTTP 429 responses continue to be treated as errors.

API documentation: https://duo.com/docs/adminapi

---------

Co-authored-by: Dan Kortschak <dan.kortschak@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Pull request that fixes a bug issue Integration:cisco_duo Cisco Duo Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

3 participants