zscaler_zia.firewall: Fix source/destination ip mapping

[kcreddy]

Proposed commit message

Fix source/destination ip mapping in firewall logs.

Currently the ECS source.ip and destination.ip are copied not only from
ZScaler client's source and destination ips, but also from the proxy server's
source and destination ips. From security detection point of view, the flow is clearly defined
if source.ip and destination.ip are mapped only from client's perspective instead of mixing
them with proxy's source and destination ips. Also, the current array representation of
source.ip and destination.ip makes it unable to be used with geoip processor.

This PR:

• Removes the mapping from proxy server's source and destination to ECS source.ip and destination.ip respectively.
• Adds source.nat.ip from zscaler's tsip field.
• Adds geoip processor to source.ip and destination.ip.
• Updates related.ip to reflect from custom fields of proxy's source and destination ips.

Reference: https://help.zscaler.com/zia/nss-feed-output-format-firewall-logs

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

Pipeline tests
cd packages/zscaler_zia && elastic-package stack down && elastic-package build && elastic-package stack up --version=8.13.0 -d -v && eval "$(elastic-package stack shellinit)" && elastic-package test pipeline --generate -v --data-streams=firewall

--- Test results for package: zscaler_zia - START ---
╭─────────────┬─────────────┬───────────┬────────────────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE     │ DATA STREAM │ TEST TYPE │ TEST NAME                                                  │ RESULT │ TIME ELAPSED │
├─────────────┼─────────────┼───────────┼────────────────────────────────────────────────────────────┼────────┼──────────────┤
│ zscaler_zia │ firewall    │ pipeline  │ (ingest pipeline warnings test-firewall-http-endpoint.log) │ PASS   │    275.157ms │
│ zscaler_zia │ firewall    │ pipeline  │ (ingest pipeline warnings test-firewall.log)               │ PASS   │ 289.361333ms │
│ zscaler_zia │ firewall    │ pipeline  │ (ingest pipeline warnings test-unicode.json)               │ PASS   │ 297.987209ms │
│ zscaler_zia │ firewall    │ pipeline  │ test-firewall-http-endpoint.log                            │ PASS   │  89.030917ms │
│ zscaler_zia │ firewall    │ pipeline  │ test-firewall.log                                          │ PASS   │  79.011458ms │
│ zscaler_zia │ firewall    │ pipeline  │ test-unicode.json                                          │ PASS   │  62.875291ms │
╰─────────────┴─────────────┴───────────┴────────────────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: zscaler_zia - END   ---
Done

System tests
cd packages/zscaler_zia && elastic-package stack down && elastic-package build && elastic-package stack up --version=8.13.0 -d -v && eval "$(elastic-package stack shellinit)" && elastic-package test system --generate -v --data-streams=firewall

--- Test results for package: zscaler_zia - START ---
╭─────────────┬─────────────┬───────────┬───────────────┬────────┬───────────────╮
│ PACKAGE     │ DATA STREAM │ TEST TYPE │ TEST NAME     │ RESULT │  TIME ELAPSED │
├─────────────┼─────────────┼───────────┼───────────────┼────────┼───────────────┤
│ zscaler_zia │ firewall    │ system    │ http-endpoint │ PASS   │ 43.568997917s │
│ zscaler_zia │ firewall    │ system    │ tcp           │ PASS   │ 46.003018042s │
╰─────────────┴─────────────┴───────────┴───────────────┴────────┴───────────────╯
--- Test results for package: zscaler_zia - END   ---
Done

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[elasticmachine]

�� Build Succeeded

• Buildkite Build
• Commit: 6fef889

cc @kcreddy

[ShourieG]

@kcreddy , Is this also part of the required change? cause I thought we were just removing the proxy address mapping from source and destination, adding geoip proc and updating related ips.

[kcreddy]

@ShourieG, yes it is also one of the asks from the SDH. I've updated the PR description.

[ShourieG]

LGTM

[elastic-vault-github-plugin-prod]

Package zscaler_zia - 3.3.0 containing this change is available at https://epr.elastic.co/search?package=zscaler_zia

[Cris-Maggi]

thank you team, I'll follow up with the customer.