x-pack/filebeat/input/cel: allow CEL programs to know remaining budget

[efd6]

Proposed commit message

x-pack/filebeat/input/cel: allow CEL programs to know remaining budget

For CEL-based integrations with a large number of pagination cycles, it
may be helpful for the program to be aware of the number of remaining
execution cycles, either to simply avoid degraded health status, or to
react differently and store cursor state for the next periodic
evaluation.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

Author's Checklist

• [ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

[github-actions]

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[github-actions]

🔍 Preview links for changed docs

• docs/reference/filebeat/filebeat-input-cel.md

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[ShourieG]

LGTM.

One query though:

In most scenarios the number of pages are not known beforehand and we iterate till size(response.body) >0 . I'm guessing this is only for scenarios where we have this number available in the response.

[efd6]

This feature does not need to be used. But I discovered a use for it specifically in the context that you describe, where the number of pages is not known. An example of how you might use it in that context is to put the current working state into the cursor on the last available iteration and the resume work when the next periodic run starts. In the case where I'm using a hand-rolled version, I'm always storing the working state in the cursor and using the number of runs relative to the maximum as a way to avoid triggering a degraded health update.

[vishaangelova]

LGTM from the docs perspective.