zscaler_zia: Fix url parsing in Web logs. - #10517
Conversation
kcreddy
added
Integration:zscaler_zia
|
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations) |
andrewkroh
added
Integration:Zscaler
| def url = ctx.zscaler_zia.web.url.name; | ||
| if (!regex.matcher(url).matches()) { | ||
| if (ctx.network?.protocol != null) { | ||
| url = ctx.network.protocol + '://' + url; |
There was a problem hiding this comment.
There may be some additional work required to cover edge cases; from the docs, the proto log field (which becomes network.protocol) is not a direct mapping to a scheme. Though maybe that should happen in the assignment to network.protocol.
There was a problem hiding this comment.
@efd6, could you please review if this seems valid:
I updated the logic to derive network.protocol, first based on alpn_protocol log field as it closely matches the ECS definition. If alpn_protocol is absent, I added a mapping to derive the network.protocol for different values of proto log field to cover edge cases.
Changes in 8ff679f.
🚀 Benchmarks reportPackage
|
| Data stream | Previous EPS | New EPS | Diff (%) | Result |
|---|---|---|---|---|
alerts |
3412.97 | 2785.52 | -627.45 (-18.38%) | 💔 |
firewall |
1490.31 | 1027.75 | -462.56 (-31.04%) | 💔 |
web |
1162.79 | 951.47 | -211.32 (-18.17%) | 💔 |
To see the full report comment with /test benchmark fullreport
| description: Set network.protocol from prototype, if not already set. | ||
| tag: script_set_network_protocol | ||
| params: | ||
| dns-over-https: dns |
There was a problem hiding this comment.
Worth having a source link for the origin of this table, though I can't find a good source that includes all the protocols that are here, so if it doesn't exist, never mind.
There was a problem hiding this comment.
I also couldn't find a mapping table, but I will add link to the list of uri schemes
|
💚 Build Succeeded
History
cc @kcreddy |
|
Package zscaler_zia - 3.0.2 containing this change is available at https://epr.elastic.co/search?package=zscaler_zia |
Fix url parsing in Web logs.
3.0.0 version introduced a bug in url parsing due to lack of url.scheme.
Earlier versions had logic to add url.scheme to `eurl` field to
generate `url.full`. When the field changed from `eurl` to `url` in 3.0.0,
this logic was removed. This leads to erroneous values in ECS `url`
fields. This PR fixes the issue.
- Add `network.protocol` as scheme to `url` field to fully
generate `url.full`.
- If `network.protocol` is missing, add default `https` as scheme.
This is required to properly run `uri_parts` processor.
4 participants
Proposed commit message
Checklist
changelog.ymlfile.How to test this PR locally