sentinel_one: expose max_executions and persist worklist in cursor - #18976
Merged
Conversation
efd6
added
enhancement
Expose max_executions as a configurable variable for the application data stream so operators can raise the execution budget for large estates where the two-level worklist traversal (inventory pages x endpoint pages) exceeds the default of 1000. Move the worklist, pagination tokens, and fetch_more flag into the cursor instead of the state root. The CEL input persists the cursor after each successful event publish, so an interrupted traversal resumes where it left off on the next interval rather than starting over from scratch. Closes elastic#18974
Contributor
🚀 Benchmarks reportPackage
|
| Data stream | Previous EPS | New EPS | Diff (%) | Result |
|---|---|---|---|---|
threat_event |
9560.23 | 6289.31 | -3270.92 (-34.21%) | 💔 |
application_risk |
12886.6 | 10604.45 | -2282.15 (-17.71%) | 💔 |
To see the full report comment with /test benchmark fullreport
💚 Build Succeeded
cc @efd6 |
efd6
marked this pull request as ready for review
May 12, 2026 23:34
|
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations) |
andrewkroh
approved these changes
May 13, 2026
andrewkroh
left a comment
andrewkroh
left a comment
Contributor
There was a problem hiding this comment.
LGTM.
Checking our OTel tracing telemetry (labels.cel_periodic_max_execution_limited:true and labels.package_name:sentinel_one) I can see there are a few agents their max_execution limits.
Contributor
|
Package sentinel_one - 2.7.0 containing this change is available at https://epr.elastic.co/package/sentinel_one/2.7.0/ |
herrBez
pushed a commit
to herrBez/integrations
that referenced
this pull request
Jun 1, 2026
…lastic#18976) Expose max_executions as a configurable variable for the application data stream so operators can raise the execution budget for large estates where the two-level worklist traversal (inventory pages x endpoint pages) exceeds the default of 1000. Move the worklist, pagination tokens, and fetch_more flag into the cursor instead of the state root. The CEL input persists the cursor after each successful event publish, so an interrupted traversal resumes where it left off on the next interval rather than starting over from scratch. Closes elastic#18974
efd6
added a commit
to efd6/integrations
that referenced
this pull request
Jul 7, 2026
…d application_risk data streams The threat_event CEL program stores its two-level worklist (threats list, page cursors, fetch_more flag) in plain state keys that are discarded on agent restart. This causes the program to re-walk the entire SentinelOne threats catalog from scratch, producing ~390x event duplication on a customer system running for 29 days after an upgrade restart. Move worklist state under state.cursor.* so the CEL input persists it to the registry. Also expose max_executions in the threat_event manifest so large estates can raise the execution budget beyond the default 1000. Apply the same cursor persistence fix to application_risk, which stores its pagination cursor in state.next.page instead of state.cursor.*. This mirrors the fix applied to the application data stream in v2.7.0 (PR elastic#18976).
efd6
added a commit
to efd6/integrations
that referenced
this pull request
Jul 7, 2026
…d application_risk data streams The threat_event CEL program stores its two-level worklist (threats list, page cursors, fetch_more flag) in plain state keys that are discarded on agent restart. This causes the program to re-walk the entire SentinelOne threats catalog from scratch, producing ~390x event duplication on a customer system running for 29 days after an upgrade restart. Move worklist state under state.cursor.* so the CEL input persists it to the registry. Also expose max_executions in the threat_event manifest so large estates can raise the execution budget beyond the default 1000. Apply the same cursor persistence fix to application_risk, which stores its pagination cursor in state.next.page instead of state.cursor.*. This mirrors the fix applied to the application data stream in v2.7.0 (PR elastic#18976).
efd6
added a commit
to efd6/integrations
that referenced
this pull request
Jul 7, 2026
…d application_risk data streams The threat_event CEL program stores its two-level worklist (threats list, page cursors, fetch_more flag) in plain state keys that are discarded on agent restart. This causes the program to re-walk the entire SentinelOne threats catalog from scratch, producing ~390x event duplication on a customer system running for 29 days after an upgrade restart. Move worklist state under state.cursor.* so the CEL input persists it to the registry. Also expose max_executions in the threat_event manifest so large estates can raise the execution budget beyond the default 1000. Apply the same cursor persistence fix to application_risk, which stores its pagination cursor in state.next.page instead of state.cursor.*. This mirrors the fix applied to the application data stream in v2.7.0 (PR elastic#18976).
efd6
added a commit
that referenced
this pull request
Jul 8, 2026
…d application_risk data streams (#20022) The threat_event CEL program stores its two-level worklist (threats list, page cursors, fetch_more flag) in plain state keys that are discarded on agent restart. This causes the program to re-walk the entire SentinelOne threats catalog from scratch. Move worklist state under state.cursor.* so the CEL input persists it to the registry. Also expose max_executions in the threat_event manifest so large estates can raise the execution budget beyond the default 1000. Apply the same cursor persistence fix to application_risk, which stores its pagination cursor in state.next.page instead of state.cursor.*. This mirrors the fix applied to the application data stream in v2.7.0 (PR #18976).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed commit message
Checklist
changelog.ymlfile.Author's Checklist
How to test this PR locally
Related issues
Screenshots