Skip to content

[Tenable_IO] - Enriched the scan data stream with WAS API scan details data. - #16222

Merged
ShourieG merged 7 commits into
elastic:mainfrom
ShourieG:tennable/scans_enrichment
Dec 10, 2025
Merged

[Tenable_IO] - Enriched the scan data stream with WAS API scan details data.#16222
ShourieG merged 7 commits into
elastic:mainfrom
ShourieG:tennable/scans_enrichment

Conversation

@ShourieG

@ShourieG ShourieG commented Dec 3, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

Type of change

  • Enhancement

Proposed commit message

tenable_io: enrich scan data stream with detailed scan information

The previous implementation only collected basic scan metadata. This change
adds a worklist pattern to fetch detailed scan information for each scan,
providing richer data for security analysis.

API documentation:
- List Scans: https://developer.tenable.com/reference/scans-list
- Scan details: https://developer.tenable.com/reference/was-v2-scans-details
   
The test for enrichment was created using the public example of scan-details available 
on https://developer.tenable.com/reference/scans-list and integrating this with an existing
list scans test.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

The tests used have been derived from the existing tests and examples listed on https://developer.tenable.com/reference/scans-list

  • [ ]

How to test this PR locally

Related issues

Screenshots
@ShourieG ShourieG added integration Label used for meta issues tracking each integration Integration:tenable_io Tenable Vulnerability Management enhancement New feature or request documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. labels Dec 3, 2025
@ShourieG ShourieG self-assigned this Dec 3, 2025
@ShourieG
ShourieG marked this pull request as ready for review December 3, 2025 11:32
@ShourieG
ShourieG requested a review from a team as a code owner December 3, 2025 11:32
@ShourieG ShourieG added the Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] label Dec 3, 2025
@elasticmachine

elasticmachine commented Dec 3, 2025

Copy link
Copy Markdown

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)
@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Dec 3, 2025

Copy link
Copy Markdown
Contributor

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport
efd6
efd6 reviewed Dec 7, 2025
View reviewed changes
Comment thread packages/tenable_io/data_stream/scan/agent/stream/cel.yml.hbs
@elasticmachine

elasticmachine commented Dec 10, 2025

Copy link
Copy Markdown

💚 Build Succeeded

History

cc @ShourieG
@ShourieG
ShourieG merged commit b127809 into elastic:main Dec 10, 2025
8 checks passed
@ShourieG
ShourieG deleted the tennable/scans_enrichment branch December 10, 2025 09:39
@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Dec 10, 2025

Copy link
Copy Markdown
Contributor

Package tenable_io - 4.7.0 containing this change is available at https://epr.elastic.co/package/tenable_io/4.7.0/
chemamartinez added a commit that referenced this pull request Aug 4, 2026
@chemamartinez
tenable_io: replace scan details endpoint and remap schema (#20347)
0880209 
tenable_io/data_stream/scan: fix scan details endpoint and remap schema

The scan data stream's CEL program fetched per-scan details via
GET /was/v2/scans/{id} (Web Application Scanning API) [1] instead of
the correct GET /scans/{id} (Vulnerability Management API) [2].
Because VM scan IDs are not valid in the WAS namespace, every Phase 2
request returned 404. The CEL 404 branch emitted a want_more stub that
the ingest pipeline dropped, so no scan documents were ever produced.
The bug was introduced in #16222; it escaped CI because the system
test mock was hand-authored to return 200 for the WAS path.

This change fixes the endpoint and rebuilds all schema-dependent
artifacts around the standard VM scan-details response:

- CEL program: /was/v2/scans/{id} → /scans/{id}.
- Fields: replace the WAS scan_details group with the full VM schema —
  info (+ acls[]), hosts, comphosts, vulnerabilities, compliance,
  history, notes, remediations, progress.
- Ingest pipeline: replace WAS ISO8601 date processors with UNIX
  processors for info timestamps and foreach processors for history[]
  dates; drop the UI-only filters[] array.
- Mock service: update test routes to /scans/{id} with standard-schema
  bodies so system tests exercise the real endpoint shape.
- Pipeline test fixture and sample event: updated to standard schema.
- Docs: update Phase 2 API reference link to scans-details.

[1] https://developer.tenable.com/reference/was-v2-scans-details
[2] https://developer.tenable.com/reference/scans-details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. enhancement New feature or request Integration:tenable_io Tenable Vulnerability Management integration Label used for meta issues tracking each integration Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

3 participants

@ShourieG @elasticmachine @efd6