Skip to content

[SSI] Remove unnecessary remove_agentless_tags pipeline processor - #20679

Merged
moxarth-rathod merged 3 commits into
elastic:mainfrom
moxarth-rathod:agentlesss-remove-org-fix
Aug 26, 2026
Merged

[SSI] Remove unnecessary remove_agentless_tags pipeline processor#20679
moxarth-rathod merged 3 commits into
elastic:mainfrom
moxarth-rathod:agentlesss-remove-org-fix

Conversation

@moxarth-rathod

@moxarth-rathod moxarth-rathod commented Aug 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Proposed commit message

Remove the `remove_agentless_tags` processor from ingest pipelines on data streams
  that only use agentless-disabled input types (`logfile`, `streaming`, `gcp-pubsub`,
  `azure-eventhub`, `aws-s3`, `gcs`, `http_endpoint`, `tcp`), where agentless never
  injects `organization`, `division`, or `team` metadata and the processor is
  unnecessary dead code.
  Affected integrations and data streams:
  - crowdstrike: falcon
  - google_scc: audit
  - m365_defender: event
  - microsoft_sentinel: event
  - panw_cortex_xdr: event
  - sublime_security: email_message
  - zscaler_zia: alerts, audit, dns, endpoint_dlp, firewall, tunnel, web
  
  Fix google_scc finding pubsub system test transform validation failures caused by
  static `eventTime` values in pubsub mock data aging out of the
  `latest_cdr_misconfigurations` and `latest_cdr_vulnerabilities` transforms'
  `@timestamp >= now-90d` filter. Bump the `gcppubsub-finding` stream publisher to
  v0.25.0, enable `--template`, and use dynamic `eventTime` values in
  `finding.log` via `{{ (now "-720h").Format ... }}` so timestamps stay within the
  90-day window on each test run.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

Related issues
@moxarth-rathod moxarth-rathod self-assigned this Aug 12, 2026
@moxarth-rathod moxarth-rathod added the enhancement New feature or request label Aug 12, 2026
@moxarth-rathod
moxarth-rathod requested a review from a team as a code owner August 12, 2026 05:58
@moxarth-rathod
moxarth-rathod requested a review from a team as a code owner August 12, 2026 05:58
@moxarth-rathod moxarth-rathod added Integration:panw_cortex_xdr Palo Alto Cortex XDR Integration:zscaler_zia Zscaler Internet Access Integration:m365_defender Microsoft Defender XDR Integration:google_scc Google Security Command Center Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] Integration:sublime_security Sublime Security Integration:microsoft_sentinel Microsoft Sentinel Team:SDE-Crest Crest developers on the Security Integrations team [elastic/sit-crest-contractors] labels Aug 12, 2026
@infra-vault-gh-plugin-prod

infra-vault-gh-plugin-prod Bot commented Aug 12, 2026

Copy link
Copy Markdown

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)
@github-actions

github-actions Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.
vera-review-bot[bot]
vera-review-bot Bot reviewed Aug 12, 2026
View reviewed changes
Comment thread packages/crowdstrike/changelog.yml Outdated
Comment thread packages/crowdstrike/changelog.yml Outdated
Comment thread packages/crowdstrike/changelog.yml Outdated
Comment thread packages/google_scc/changelog.yml Outdated
Comment thread packages/m365_defender/changelog.yml Outdated
Comment thread packages/microsoft_sentinel/changelog.yml Outdated
Comment thread packages/panw_cortex_xdr/changelog.yml Outdated
Comment thread packages/sublime_security/changelog.yml Outdated
Comment thread packages/zscaler_zia/changelog.yml Outdated
@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Aug 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🚀 Benchmarks report

Package crowdstrike 👍(7) 💚(5) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
fdr 1927.9 1615.51 -312.39 (-16.2%) 💔
falcon 6211.18 2997.6 -3213.58 (-51.74%) 💔

Package google_scc 👍(3) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
audit 3460.21 2202.64 -1257.57 (-36.34%) 💔

Package m365_defender 👍(5) 💚(1) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
incident 1179.25 798.4 -380.85 (-32.3%) 💔
vulnerability 10309.28 6277.46 -4031.82 (-39.11%) 💔

Package microsoft_sentinel 👍(2) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
incident 4310.34 3125 -1185.34 (-27.5%) 💔

Package sublime_security 👍(0) 💚(1) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
email_message 1112.35 771.6 -340.75 (-30.63%) 💔
message_event 9615.38 7812.5 -1802.88 (-18.75%) 💔

Package zscaler_zia 👍(9) 💚(0) 💔(3)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
sandbox_verdict 7751.94 5617.98 -2133.96 (-27.53%) 💔
firewall 1779.36 1191.9 -587.46 (-33.02%) 💔
saas_security 3105.59 1941.75 -1163.84 (-37.48%) 💔

To see the full report comment with /test benchmark fullreport
efd6
efd6 reviewed Aug 12, 2026
View reviewed changes

@efd6 efd6 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after conflicts are resolved.
@moxarth-rathod
moxarth-rathod requested a review from efd6 August 17, 2026 08:27
@moxarth-rathod
moxarth-rathod force-pushed the agentlesss-remove-org-fix branch from 8c5036b to fe3993f Compare August 19, 2026 12:53
@vera-review-bot

vera-review-bot Bot commented Aug 21, 2026

Copy link
Copy Markdown

🟢 No issues across the latest commits 9e06ca0…dd85ec0 (29 commits).

Review summary

Issues found across earlier commits 4b43445…fe3993f (33 commits) — 1 low

Package-level:

  • 🔵 packages/crowdstrike/data_stream/fdr/elasticsearch/ingest_pipeline/default.yml keeps the same dead Agentless-metadata remove processor
Issues found across earlier commits 379ead4 — 8 medium, 1 low
  • 🟡 Every new changelog entry links to the placeholder PR pull/1 instead of this PR (link) (Resolved)
  • 🔵 Each package takes a minor bump with type enhancement for a change with no runtime effect (link) (Resolved)
  • 🟡 Changelog links a different PR number (link) (Resolved)
  • 🟡 Changelog links a different PR number (link) (Resolved)
  • 🟡 Changelog links a different PR number (link) (Resolved)
  • 🟡 Changelog links a different PR number (link) (Resolved)
  • 🟡 Changelog links a different PR number (link) (Resolved)
  • 🟡 Changelog links a different PR number (link) (Resolved)
  • 🟡 Changelog links a different PR number (link) (Resolved)

A new commit triggers another review — at most once every 15 minutes. I skip the PR while it's approved or has merge conflicts.

🤖 AI-Generated Review | Vera Review Bot - v0.2.6 | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.
efd6
efd6 approved these changes Aug 26, 2026
View reviewed changes

@efd6 efd6 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after conflict is resolved.
@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

✅ All changelog entries have the correct PR link.
@infra-vault-gh-plugin-prod

infra-vault-gh-plugin-prod Bot commented Aug 26, 2026

Copy link
Copy Markdown

💚 Build Succeeded

History

cc @moxarth-rathod
@mergify

mergify Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request
@moxarth-rathod
moxarth-rathod merged commit 16fe0ed into elastic:main Aug 26, 2026
9 checks passed
@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Package crowdstrike - 4.9.0 containing this change is available at https://epr.elastic.co/package/crowdstrike/4.9.0/
@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Package google_scc - 2.8.0 containing this change is available at https://epr.elastic.co/package/google_scc/2.8.0/
@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Package m365_defender - 5.18.0 containing this change is available at https://epr.elastic.co/package/m365_defender/5.18.0/
@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Package microsoft_sentinel - 1.6.0 containing this change is available at https://epr.elastic.co/package/microsoft_sentinel/1.6.0/
@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Package panw_cortex_xdr - 2.9.0 containing this change is available at https://epr.elastic.co/package/panw_cortex_xdr/2.9.0/
@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Package sublime_security - 1.15.0 containing this change is available at https://epr.elastic.co/package/sublime_security/1.15.0/
@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Package zscaler_zia - 4.4.0 containing this change is available at https://epr.elastic.co/package/zscaler_zia/4.4.0/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Integration:crowdstrike CrowdStrike Integration:google_scc Google Security Command Center Integration:m365_defender Microsoft Defender XDR Integration:microsoft_sentinel Microsoft Sentinel Integration:panw_cortex_xdr Palo Alto Cortex XDR Integration:sublime_security Sublime Security Integration:zscaler_zia Zscaler Internet Access Team:SDE-Crest Crest developers on the Security Integrations team [elastic/sit-crest-contractors] Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

2 participants

@moxarth-rathod @efd6