Skip to content

tenable_io: fix flakey test and timestamp handling - #10940

Merged
efd6 merged 6 commits into
elastic:mainfrom
efd6:10626-tenable_io
Sep 13, 2024
Merged

tenable_io: fix flakey test and timestamp handling#10940
efd6 merged 6 commits into
elastic:mainfrom
efd6:10626-tenable_io

Conversation

@efd6

@efd6 efd6 commented Aug 30, 2024

Copy link
Copy Markdown
Contributor

Proposed commit message

The comparison for determining whether the pagination had completed was based
on the response's size and the batch size (via state.batch_size, but could
also have been the response's params.size without difference in behaviour).
This was not correct since the size value is the current response's result
set and so may match the batch size even when the pagination is complete, in
the case that total size mod batch size is zero. Instead keep a running tally
of records returned throughout the pagination and compare this to the total
count reported by the API. Also fix the test case so that the response
matches the API docs.[1]

The timestamps were being examined for their maximum by string comparison.
This should approximately always work if the timestamp is formatted as
RFC3339, but the timestamps are rendered as variable precision RFC3339, so
make sure that the comparisons work by converting to timestamp values to
obtain the maximum.

[1]https://developer.tenable.com/reference/io-plugins-list

I don't now how it was working at all.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@efd6 efd6 self-assigned this Aug 30, 2024
@efd6 efd6 added Integration:tenable_io Tenable Vulnerability Management bugfix Pull request that fixes a bug issue Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Aug 30, 2024
@efd6
efd6 force-pushed the 10626-tenable_io branch 2 times, most recently from 62867fb to a98fc94 Compare August 30, 2024 04:17
@efd6
efd6 force-pushed the 10626-tenable_io branch from a98fc94 to e3ff7cc Compare August 30, 2024 04:19
@efd6
efd6 force-pushed the 10626-tenable_io branch from e3ff7cc to ed82660 Compare August 30, 2024 04:20
@elasticmachine

Copy link
Copy Markdown

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@elasticmachine

Copy link
Copy Markdown

💚 Build Succeeded

History

  • 💔 Build #15216 failed 62867fbd272b8e61b311d3b2d2418e701b21718a

cc @efd6

@efd6
efd6 marked this pull request as ready for review August 30, 2024 04:45
@efd6
efd6 requested a review from a team as a code owner August 30, 2024 04:45
@elasticmachine

Copy link
Copy Markdown

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@efd6
efd6 merged commit 08e244c into elastic:main Sep 13, 2024
@elasticmachine

Copy link
Copy Markdown

Package tenable_io - 3.2.1 containing this change is available at https://epr.elastic.co/search?package=tenable_io

harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 4, 2025
The comparison for determining whether the pagination had completed was based
on the response's size and the batch size (via state.batch_size, but could
also have been the response's params.size without difference in behaviour).
This was not correct since the size value is the current response's result
set and so may match the batch size even when the pagination is complete, in
the case that total size mod batch size is zero. Instead keep a running tally
of records returned throughout the pagination and compare this to the total
count reported by the API. Also fix the test case so that the response
matches the API docs.[1]

The timestamps were being examined for their maximum by string comparison.
This should approximately always work if the timestamp is formatted as
RFC3339, but the timestamps are rendered as variable precision RFC3339, so
make sure that the comparisons work by converting to timestamp values to
obtain the maximum.

[1]https://developer.tenable.com/reference/io-plugins-list
harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 5, 2025
The comparison for determining whether the pagination had completed was based
on the response's size and the batch size (via state.batch_size, but could
also have been the response's params.size without difference in behaviour).
This was not correct since the size value is the current response's result
set and so may match the batch size even when the pagination is complete, in
the case that total size mod batch size is zero. Instead keep a running tally
of records returned throughout the pagination and compare this to the total
count reported by the API. Also fix the test case so that the response
matches the API docs.[1]

The timestamps were being examined for their maximum by string comparison.
This should approximately always work if the timestamp is formatted as
RFC3339, but the timestamps are rendered as variable precision RFC3339, so
make sure that the comparisons work by converting to timestamp values to
obtain the maximum.

[1]https://developer.tenable.com/reference/io-plugins-list
@efd6
efd6 deleted the 10626-tenable_io branch February 5, 2025 22:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Pull request that fixes a bug issue Integration:tenable_io Tenable Vulnerability Management Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

3 participants