Skip to content

[carbon_black_cloud] Fix alert_v7 CEL pagination logic - #11259

Merged
chrisberkhout merged 2 commits into
elastic:mainfrom
chrisberkhout:carbon_black_cloud-alert_v7-pagination
Sep 27, 2024
Merged

[carbon_black_cloud] Fix alert_v7 CEL pagination logic#11259
chrisberkhout merged 2 commits into
elastic:mainfrom
chrisberkhout:carbon_black_cloud-alert_v7-pagination

Conversation

@chrisberkhout

@chrisberkhout chrisberkhout commented Sep 26, 2024

Copy link
Copy Markdown
Contributor

Proposed commit message

[carbon_black_cloud] Fix alert_v7 CEL pagination logic

When using the CEL input for the `alert_v7` data stream, data would
load very slowly for two reason:

- The page size defaulted to 20 results.
- The `want_more` flag was incorrectly set to false.

The response body has the fields `num_found` and `num_available`.
The `num_found` field gives the total number of results matching the query.
The `num_available` filed is equal to `min(num_found, 10000)`.

Now `want_more` is true if `num_found` is less than the page size, so it
will keep fetching more until less than a full page of results are found.

The page size is increased from 20 (the default) to 100.

Testing

I tested this manually.

When no results are found, the num_found field is returned with the value 0, so the pagination condition can assume it is always there.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
@chrisberkhout chrisberkhout added Integration:carbon_black_cloud VMware Carbon Black Cloud bugfix Pull request that fixes a bug issue Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Sep 26, 2024
@chrisberkhout chrisberkhout self-assigned this Sep 26, 2024
@chrisberkhout
chrisberkhout requested a review from a team as a code owner September 26, 2024 18:26
@elasticmachine

Copy link
Copy Markdown

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@elasticmachine

Copy link
Copy Markdown

���� Build Succeeded

cc @chrisberkhout

@chrisberkhout
chrisberkhout merged commit de85109 into elastic:main Sep 27, 2024
@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package carbon_black_cloud - 2.5.4 containing this change is available at https://epr.elastic.co/search?package=carbon_black_cloud

@chrisberkhout
chrisberkhout deleted the carbon_black_cloud-alert_v7-pagination branch September 27, 2024 08:14
harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 4, 2025
When using the CEL input for the `alert_v7` data stream, data would
load very slowly for two reason:

- The page size defaulted to 20 results.
- The `want_more` flag was incorrectly set to false.

The response body has the fields `num_found` and `num_available`.
The `num_found` field gives the total number of results matching the query.
The `num_available` filed is equal to `min(num_found, 10000)`.

Now `want_more` is true if `num_found` is less than the page size, so it
will keep fetching more until less than a full page of results are found.

The page size is increased from 20 (the default) to 100.
harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 5, 2025
When using the CEL input for the `alert_v7` data stream, data would
load very slowly for two reason:

- The page size defaulted to 20 results.
- The `want_more` flag was incorrectly set to false.

The response body has the fields `num_found` and `num_available`.
The `num_found` field gives the total number of results matching the query.
The `num_available` filed is equal to `min(num_found, 10000)`.

Now `want_more` is true if `num_found` is less than the page size, so it
will keep fetching more until less than a full page of results are found.

The page size is increased from 20 (the default) to 100.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Pull request that fixes a bug issue Integration:carbon_black_cloud VMware Carbon Black Cloud Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

3 participants