Skip to content

[zscaler_zia] Add support for SaaS Security Activity data stream - #18986

Merged
brijesh-elastic merged 8 commits into
elastic:mainfrom
brijesh-elastic:zscaler_zia-saas_security_activity
Jun 4, 2026
Merged

[zscaler_zia] Add support for SaaS Security Activity data stream#18986
brijesh-elastic merged 8 commits into
elastic:mainfrom
brijesh-elastic:zscaler_zia-saas_security_activity

Conversation

@brijesh-elastic

@brijesh-elastic brijesh-elastic commented May 13, 2026

Copy link
Copy Markdown
Contributor

Proposed commit message

zscaler_zia: Add support for SaaS Security Activity data stream.

This data stream collects SaaS Security Activity logs[1] using Zscaler ZIA
Nanolog Streaming Service (NSS).

Test samples were derived from documentation, 
which were subsequently sanitized.

[1] https://help.zscaler.com/zia/nss-feed-output-format-saas-security-activity-logs

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

  • Clone integrations repo.
  • Install elastic package locally.
  • Start elastic stack using elastic-package.
  • Move to integrations/packages/zscaler_zia directory.
  • Run the following command to run tests.

elastic-package test -v
@brijesh-elastic brijesh-elastic self-assigned this May 13, 2026
@brijesh-elastic brijesh-elastic added documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. enhancement New feature or request Integration:zscaler_zia Zscaler Internet Access Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] Team:SDE-Crest Crest developers on the Security Integrations team [elastic/sit-crest-contractors] labels May 13, 2026
@github-actions

github-actions Bot commented May 13, 2026

Copy link
Copy Markdown
Contributor

Vale Linting Results

Summary: 6 warnings found

⚠️ Warnings (6)
File Line Rule Message
packages/zscaler_zia/docs/README.md 1376 Elastic.Latinisms Latin terms and abbreviations are a common source of confusion. Use 'and so on' instead of 'etc'.
packages/zscaler_zia/docs/README.md 1376 Elastic.DirectionalLanguage Don't use directional language. Use 'earlier on this page' instead of 'noted above'.
packages/zscaler_zia/docs/README.md 1377 Elastic.DirectionalLanguage Don't use directional language. Use 'earlier on this page' instead of 'noted above'.
packages/zscaler_zia/docs/README.md 1378 Elastic.QuotesPunctuation Place punctuation inside closing quotation marks.
packages/zscaler_zia/docs/README.md 1379 Elastic.Latinisms Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/zscaler_zia/docs/README.md 1380 Elastic.Latinisms Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.

The Vale linter checks documentation changes against the Elastic Docs style guide.

To use Vale locally or report issues, refer to Elastic style guide for Vale.
@github-actions github-actions Bot mentioned this pull request May 13, 2026
Merged
7 tasks
@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented May 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🚀 Benchmarks report

Package zscaler_zia 👍(7) 💚(2) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
endpoint_dlp 2427.18 1751.31 -675.87 (-27.85%) 💔

To see the full report comment with /test benchmark fullreport
@brijesh-elastic brijesh-elastic added the dashboard Relates to a Kibana dashboard bug, enhancement, or modification. label May 26, 2026
@brijesh-elastic
brijesh-elastic marked this pull request as ready for review May 26, 2026 07:22
@brijesh-elastic
brijesh-elastic requested review from a team as code owners May 26, 2026 07:22
@infra-vault-gh-plugin-prod

infra-vault-gh-plugin-prod Bot commented May 26, 2026

Copy link
Copy Markdown

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)
navnit-elastic
navnit-elastic reviewed May 26, 2026
View reviewed changes
Comment thread ...ges/zscaler_zia/data_stream/saas_security_activity/elasticsearch/ingest_pipeline/default.yml Outdated
Comment thread ...ges/zscaler_zia/data_stream/saas_security_activity/elasticsearch/ingest_pipeline/default.yml
Comment thread ...ges/zscaler_zia/data_stream/saas_security_activity/elasticsearch/ingest_pipeline/default.yml
Comment thread ...ges/zscaler_zia/data_stream/saas_security_activity/elasticsearch/ingest_pipeline/default.yml Outdated
field: event.kind
tag: set_event_kind_to_event
value: event
- append:

@navnit-elastic navnit-elastic May 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think event.category: iam is over-broad if applied to all activity types. We should map activity.type to event.category if we know all possible values for activity.type, e.g. append file for file ops and authentication for Login.

@brijesh-elastic brijesh-elastic May 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in e17159d
@github-actions github-actions Bot mentioned this pull request May 27, 2026
Closed
5 tasks
efd6
efd6 reviewed Jun 1, 2026
View reviewed changes
Comment thread packages/zscaler_zia/data_stream/saas_security_activity/agent/stream/http_endpoint.yml.hbs Outdated
Comment on lines +3 to +6
{{#if enable_request_tracer}}
tracer.filename: "../../logs/http_endpoint/http-request-trace-*.ndjson"
tracer.maxbackups: 5
{{/if}}

@efd6 efd6 Jun 1, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be using the 8.15+ form (ref).

If there are other instances of this in this package, please fix them too.

@brijesh-elastic brijesh-elastic Jun 2, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, for the other data streams, it is being handled in #19333
Comment thread packages/zscaler_zia/data_stream/saas_security_activity/manifest.yml Outdated
type: bool
multi: false
default: false
- name: preserve_duplicate_custom_fields

@efd6 efd6 Jun 1, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer recommended.

@brijesh-elastic brijesh-elastic Jun 1, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

meaning?

@efd6 efd6 Jun 1, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not be adding it to new data streams.

@brijesh-elastic brijesh-elastic Jun 1, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, are you suggesting that we completely remove the preserve_duplicate_custom_fields variable?

@efd6 efd6 Jun 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes
@brijesh-elastic brijesh-elastic mentioned this pull request Jun 2, 2026
Merged
5 tasks
@brijesh-elastic
brijesh-elastic requested a review from efd6 June 2, 2026 07:13
efd6
efd6 reviewed Jun 2, 2026
View reviewed changes

@efd6 efd6 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after conflicts are fixed.
@brijesh-elastic
brijesh-elastic requested a review from efd6 June 3, 2026 06:27
@elasticmachine

elasticmachine commented Jun 3, 2026

Copy link
Copy Markdown

💚 Build Succeeded

History

cc @brijesh-elastic
@brijesh-elastic
brijesh-elastic merged commit c792809 into elastic:main Jun 4, 2026
12 checks passed
@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Package zscaler_zia - 3.21.0 containing this change is available at https://epr.elastic.co/package/zscaler_zia/3.21.0/
@github-actions github-actions Bot mentioned this pull request Jun 4, 2026
Closed
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dashboard Relates to a Kibana dashboard bug, enhancement, or modification. documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. enhancement New feature or request Integration:zscaler_zia Zscaler Internet Access Team:SDE-Crest Crest developers on the Security Integrations team [elastic/sit-crest-contractors] Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

4 participants

@brijesh-elastic @elasticmachine @efd6 @navnit-elastic