Skip to content

zscaler_zia: Add hex-encoded fields in web logs. - #10814

Merged
kcreddy merged 3 commits into
elastic:mainfrom
kcreddy:zscaler_zia_web_hexfields
Aug 20, 2024
Merged

zscaler_zia: Add hex-encoded fields in web logs.#10814
kcreddy merged 3 commits into
elastic:mainfrom
kcreddy:zscaler_zia_web_hexfields

Conversation

@kcreddy

@kcreddy kcreddy commented Aug 19, 2024

Copy link
Copy Markdown
Contributor

Proposed commit message

Add hex-encoded fields in web logs.

ZIA exposes hex-encoded fields to allow SIEM to parse the control characters
present inside fields. Adding these fields instead of non-encoded fields allow
for proper parsing of events.

  • Existing rename processors to convert non-encoded fields
    are updated to urldecode processors to decode the hex-encoded fields.
  • Fields such as erefererpath, eurlpath, and edepartment that were
    missing from previous template are added.
  • Updated web template version v7.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

cd packages/zscaler_zia && elastic-package stack down && elastic-package build && elastic-package stack up --version=8.13.0 -d -v && eval "$(elastic-package stack shellinit)" && elastic-package test pipeline --generate -v --data-streams=web

--- Test results for package: zscaler_zia - START ---
╭─────────────┬─────────────┬───────────┬───────────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE     │ DATA STREAM │ TEST TYPE │ TEST NAME                                             │ RESULT │ TIME ELAPSED │
├─────────────┼─────────────┼───────────┼───────────────────────────────────────────────────────┼────────┼──────────────┤
│ zscaler_zia │ web         │ pipeline  │ (ingest pipeline warnings test-web-http-endpoint.log) │ PASS   │ 461.119417ms │
│ zscaler_zia │ web         │ pipeline  │ (ingest pipeline warnings test-web.log)               │ PASS   │ 548.698209ms │
│ zscaler_zia │ web         │ pipeline  │ test-web-http-endpoint.log                            │ PASS   │ 123.961417ms │
│ zscaler_zia │ web         │ pipeline  │ test-web.log                                          │ PASS   │ 237.730667ms │
╰─────────────┴─────────────┴───────────┴───────────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: zscaler_zia - END   ---
Done
@andrewkroh andrewkroh added the Integration:zscaler_zia Zscaler Internet Access label Aug 19, 2024
@kcreddy kcreddy added enhancement New feature or request Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Aug 19, 2024
@elasticmachine

Copy link
Copy Markdown

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@elasticmachine

Copy link
Copy Markdown

💚 Build Succeeded

@kcreddy
kcreddy marked this pull request as ready for review August 20, 2024 03:23
@kcreddy
kcreddy requested a review from a team as a code owner August 20, 2024 03:23
@elasticmachine

Copy link
Copy Markdown

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@kcreddy kcreddy self-assigned this Aug 20, 2024

@ShourieG ShourieG left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kcreddy
kcreddy merged commit 959ae90 into elastic:main Aug 20, 2024
@elasticmachine

Copy link
Copy Markdown

Package zscaler_zia - 3.2.0 containing this change is available at https://epr.elastic.co/search?package=zscaler_zia

harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 4, 2025
Add hex-encoded fields in web logs.

ZIA exposes hex-encoded fields to allow SIEM to parse the control characters 
present inside fields. Adding these fields instead of non-encoded fields allow 
for proper parsing of events. 

   - Existing `rename` processors to convert non-encoded fields
are updated to `urldecode` processors to decode the hex-encoded fields.  
   - Fields such as `erefererpath`, `eurlpath`, and `edepartment` that were 
      missing from previous template are added.
   - Updated web template version `v7`.
harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 5, 2025
Add hex-encoded fields in web logs.

ZIA exposes hex-encoded fields to allow SIEM to parse the control characters 
present inside fields. Adding these fields instead of non-encoded fields allow 
for proper parsing of events. 

   - Existing `rename` processors to convert non-encoded fields
are updated to `urldecode` processors to decode the hex-encoded fields.  
   - Fields such as `erefererpath`, `eurlpath`, and `edepartment` that were 
      missing from previous template are added.
   - Updated web template version `v7`.
@kcreddy
kcreddy deleted the zscaler_zia_web_hexfields branch February 7, 2025 09:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Integration:zscaler_zia Zscaler Internet Access Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

4 participants