[cisco_ios] Fix IPACCESSLOGP parse when MAC precedes source IP. - #20392
[cisco_ios] Fix IPACCESSLOGP parse when MAC precedes source IP.#20392ie-ops wants to merge 4 commits into
Conversation
ie-ops
added
bugfix
✅ Elastic Docs Style Checker (Vale)No issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
🚀 Benchmarks reportTo see the full report comment with |
|
Pinging @elastic/integration-experience (Team:Integration-Experience) |
![vera-review-bot[bot]](https://cdn.statically.io/img/avatars.githubusercontent.com/in/3297022?s=60&v=4){kind=small}
| field: source.address | ||
| tag: grok_strip_mac_from_source_address | ||
| patterns: | ||
| - '^%{MAC:source.mac} %{IP:source.address}$' |
There was a problem hiding this comment.
Severity: 🟡 Medium confidence: medium path: packages/cisco_ios/data_stream/log/elasticsearch/ingest_pipeline/default.yml:293
The new grok accepts Cisco's dotted MAC notation (0050.56b1.2233) but the pipeline only normalizes colon-separated MACs, so source.mac gets indexed in non-ECS format. Add a gsub that rewrites the dotted triple form to RFC 7042 dashes.
Details
Grok's %{MAC} pattern is defined as (?:%{CISCOMAC}|%{WINDOWSMAC}|%{COMMONMAC}), so this processor captures Cisco's dotted-triple notation (e.g. 0050.56b1.2233) into source.mac. The only MAC normalization in this pipeline is gsub_source_mac_328298a4 (line 552), which replaces ':' with '-', followed by uppercase_source_mac_5b4e7be2 (line 558). Neither touches dots, so a dotted capture is indexed as '0050.56B1.2233'. ECS requires the RFC 7042 dash-separated form, and this package documents exactly that for source.mac (docs/README.md:471). This is the notation Cisco IOS actually emits for MAC addresses — see this package's own fixtures: test-fqdn.log uses 0015.5d9c.3d01 and 0000.a636.6867, test-syslog.log uses 001e.0b80.13b5. So the very format the new processor is most likely to encounter in production is the one that ends up malformed.
Recommendation:
Add a gsub that converts the Cisco dotted-triple form before the existing uppercase step (place it next to gsub_source_mac_328298a4 around line 552):
- gsub:
tag: gsub_source_mac_cisco_dotted_1a2b3c4d
field: source.mac
pattern: '^([0-9A-Fa-f]{2})([0-9A-Fa-f]{2})\.([0-9A-Fa-f]{2})([0-9A-Fa-f]{2})\.([0-9A-Fa-f]{2})([0-9A-Fa-f]{2})$'
replacement: '$1-$2-$3-$4-$5-$6'
ignore_missing: true
- gsub:
tag: gsub_source_mac_328298a4
field: source.mac
pattern: ':'
replacement: '-'
ignore_missing: true🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| tag: dissect_gnp | ||
| pattern: "list %{cisco.ios.access_list} %{_temp_.event.action} %{network.iana_number} %{source.address} %{} %{destination.address}, %{source.packets} packet" | ||
| if: "['IPACCESSLOGNP', 'ACCESSLOGNP'].contains(ctx.event?.code)" | ||
| - grok: |
There was a problem hiding this comment.
Severity: 🔵 Low confidence: high path: packages/cisco_ios/data_stream/log/elasticsearch/ingest_pipeline/default.yml:289
The same FMANFP log line also carries the ingress interface (TenGigabitEthernet0/1/1), which dissect_gp discards into its %{} placeholder; capture it into cisco.ios.interface.name instead of dropping it.
Details
In the new fixture line the token between the source port and -> is the ingress interface name (TenGigabitEthernet0/1/1), not the -> arrow. dissect_gp (tag dissect_gp) maps that token to the anonymous %{} key, so it is silently dropped: the added expected document contains no interface field even though the interface is present in event.original. The package already defines cisco.ios.interface.name (data_stream/log/fields/fields.yml) and populates it for the FW DROP_PKT and SESS_AUDIT_TRAIL messages, so the field and mapping already exist. Since this PR is what makes the IOS-XE FMANFP variant parse correctly, this is the natural place to also retain the interface. Note the older FMANFP form (...(59144) -> 10.100.8.34(1103)) has a bare -> in that position, so the extraction must not fire for it.
Recommendation:
Add a conditional grok after the ACL dissect processors that only matches the interface-prefixed form (the \S+ before -> cannot match the bare -> of the older format):
- grok:
field: message
tag: grok_fmanfp_ingress_interface
description: Capture the ingress interface IOS-XE places before the '->' in FMANFP ACL logs.
patterns:
- '\)\s+%{NOTSPACE:cisco.ios.interface.name}->\s'
if: "['IPACCESSLOGP', 'ACCESSLOGP', 'IPV6ACCESSLOGP'].contains(ctx.event?.code)"
ignore_missing: true
ignore_failure: trueThen add the resulting cisco.ios.interface.name value to the new entry in test-cisco-ios.log-expected.json.
🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| tag: grok_strip_mac_from_source_address | ||
| description: Strip the L2 MAC address that IOS-XE prepends to the source IP in FMANFP ACL logs. | ||
| patterns: | ||
| - '^%{MAC:source.mac} %{IP:source.address}$' |
There was a problem hiding this comment.
Severity: 🔵 Low confidence: low path: packages/cisco_ios/data_stream/log/elasticsearch/ingest_pipeline/default.yml:294
The MAC strip is anchored to ^MAC IP$, so it only repairs the port-bearing ACL variants; consider covering the space-delimited dissects (IPACCESSLOGDP/RP/SP/NP) that the same FMANFP facility emits.
Details
The new grok only rewrites source.address when its whole value is <MAC> <IP>. That shape is produced only by dissect_gp / dissect_gp_access_list, whose patterns terminate source.address at the ( of the source port, so the MAC and IP end up in the same key. The sibling dissects for the same FMANFP facility - dissect_gdp (IPACCESSLOGDP, already exercised by fixture lines 25-26 of test-cisco-ios.log), dissect_grp, dissect_gsp, dissect_gnp, dissect_sp - terminate source.address at a space instead. With an L2 MAC prepended they assign only the MAC to source.address, the real source IP to the anonymous %{} key, and -> <dst> to destination.address, so both endpoints are wrong and the new grok never fires. If the MAC prefix is emitted by the ACL logging layer rather than only by the protocol/port message format, those variants need the same treatment. Flagging at low confidence because this PR ships no fixture for a MAC-prefixed DP/RP/SP/NP line, so the vendor behaviour for those codes is not demonstrated here.
Recommendation:
If IOS-XE also prepends the MAC on the space-delimited ACL messages, make the dissects tolerate it rather than relying on the anchored grok, e.g. capture the MAC explicitly in a dedicated dissect that runs first:
- dissect:
field: message
tag: dissect_gdp_with_mac
pattern: "list %{cisco.ios.access_list} %{_temp_.event.action} %{network.transport} %{source.mac} %{source.address} %{} %{destination.address} (%{icmp.type}/%{icmp.code}), %{source.packets} packet"
if: "['IPACCESSLOGDP', 'ACCESSLOGDP'].contains(ctx.event?.code)"
ignore_failure: trueand add a MAC-prefixed IPACCESSLOGDP line to test-cisco-ios.log so the behaviour is pinned by a test.
🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
|
✅ All changelog entries have the correct PR link. |
| - '^%{MAC:source.mac} %{IP:source.address}$' | ||
| if: >- | ||
| ['IPACCESSLOGP', 'ACCESSLOGP', 'IPV6ACCESSLOGP', | ||
| 'IPACCESSLOGNP', 'ACCESSLOGNP', |
There was a problem hiding this comment.
Severity: 🟡 Medium confidence: high path: packages/cisco_ios/data_stream/log/elasticsearch/ingest_pipeline/default.yml:297
The condition lists IPACCESSLOGNP/ACCESSLOGNP/IPACCESSLOGSP/ACCESSLOGSP/IPACCESSLOGRP, but the dissect patterns for those codes split source.address on whitespace so this grok can never match for them. Narrow the condition to the port-bearing codes, and handle the MAC-prefixed form of the space-delimited variants in their own dissect patterns instead.
Details
grok_strip_mac_from_source_address only fires when source.address holds the two-token value ' '. That shape is produced exclusively by dissect_gp (line 252) and dissect_gp_access_list (line 258), whose patterns terminate %{source.address} on the literal '(' of the port, so the MAC and the IP land in the same capture.
The dissects for the other five codes named in this condition use a space as the delimiter: dissect_grp (line 269, IPACCESSLOGRP), dissect_gsp (line 274, IPACCESSLOGSP), dissect_sp (line 279, ACCESSLOGSP) and dissect_gnp (line 284, IPACCESSLOGNP/ACCESSLOGNP) all match 'list %{acl} %{action} %{proto} %{source.address} %{} %{destination.address}, ...'. On a MAC-prefixed line such as 'list 177 denied igmp 0000.5e00.5323 192.168.100.197 -> 224.0.0.22, 1 packet' that assigns source.address='0000.5e00.5323', discards the real source IP into the %{} placeholder, and leaves destination.address='-> 224.0.0.22' (which then fails convert_destination_ip and is stored as destination.domain).
So source.address never contains a space for those five codes, the anchored '^%{MAC} %{IP}$' pattern cannot match, and the processor is a no-op for them. Adding the codes to the condition gives the appearance of coverage without changing behaviour, and the MAC-prefixed form of those message types remains mis-parsed. There is also no fixture in test-cisco-ios.log exercising any of the five, so nothing in the test suite contradicts this.
Recommendation:
Restrict the condition to the codes whose dissects can actually yield ' ':
- grok:
field: source.address
tag: grok_strip_mac_from_source_address
description: Strip the L2 MAC address that IOS-XE prepends to the source IP in FMANFP ACL logs.
patterns:
- '^%{MAC:source.mac} %{IP:source.address}$'
if: "['IPACCESSLOGP', 'ACCESSLOGP', 'IPV6ACCESSLOGP'].contains(ctx.event?.code)"
ignore_failure: true
ignore_missing: trueIf the space-delimited variants also need to support the MAC prefix, that has to be fixed where the split happens, by adding a MAC-aware pattern ahead of the existing dissect for each code and gating it on the MAC actually being present, for example:
- grok:
field: message
tag: grok_grp_with_mac
patterns:
- '^list %{NOTSPACE:cisco.ios.access_list} %{WORD:_temp_.event.action} %{NOTSPACE:network.transport} %{MAC:source.mac} %{IP:source.address} -> %{IP:destination.address}, %{NUMBER:source.packets} packet'
if: "ctx.event?.code == 'IPACCESSLOGRP'"
ignore_failure: trueand a matching pipeline fixture line so the branch is covered.
🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| # newer versions go on top | ||
| - version: "1.36.1" | ||
| changes: | ||
| - description: Fix FMANFP IPACCESSLOGP parsing when Cisco IOS prepends an L2 MAC address before the source IP. |
There was a problem hiding this comment.
Severity: 🔵 Low confidence: medium path: packages/cisco_ios/changelog.yml:4
The changelog entry only mentions the MAC-prefix fix, but this release also starts populating cisco.ios.interface.name from FMANFP ACL logs and normalising Cisco dotted-triple MACs. Add changelog entries for those two user-visible changes.
Details
Two of the three pipeline changes in this PR are not described in the changelog. grok_fmanfp_ingress_interface (default.yml:302) newly populates cisco.ios.interface.name for IPACCESSLOGP/ACCESSLOGP/IPV6ACCESSLOGP events, which is a field that previously only appeared on FW SESS_AUDIT_TRAIL and DROP_PKT events. gsub_source_mac_cisco_dotted_1a2b3c4d (default.yml:566) newly rewrites Cisco dotted-triple MAC notation into the RFC 7042 dash form, changing the indexed value of source.mac for any event whose MAC arrives in dotted form. Users reading the changelog to decide whether to upgrade, or debugging why a new field appeared, get no signal about either.
Recommendation:
Extend the 1.36.1 entry to cover all three behaviour changes:
- version: "1.36.1"
changes:
- description: Fix FMANFP IPACCESSLOGP parsing when Cisco IOS prepends an L2 MAC address before the source IP.
type: bugfix
link: https://github.com/elastic/integrations/pull/20392
- description: Populate `cisco.ios.interface.name` from the ingress interface in FMANFP ACL log messages.
type: enhancement
link: https://github.com/elastic/integrations/pull/20392
- description: Normalize Cisco dotted-triple MAC addresses in `source.mac` to the ECS dash-separated format.
type: enhancement
link: https://github.com/elastic/integrations/pull/20392🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
Review summaryIssues found across the latest commits f4b145b — 1 medium, 1 low
Issues found across earlier commits 36e674f — 2 lowIssues found across earlier commits 0d22ed8 — 2 medium, 1 low
🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
|
💚 Build Succeeded
History
|
| { | ||
| "expected": [ | ||
| { | ||
| "@timestamp": "2026-05-22T11:39:00.000Z", |
There was a problem hiding this comment.
@haetamoudi quick question, why are we losing these in the expected files?
3 participants
Executive summary
The Cisco IOS FMANFP IPACCESSLOGP ingest pipeline failed with an 'is not an IP string literal' error when a Cisco IOS device prepended an L2 MAC address (e.g.
00-00-5E-00-53-23) before the source IP in the log line. A new grok processor is inserted after the IPACCESSLOGNP/ACCESSLOGNP dissect step to detect and strip any leading MAC address fromsource.address, relocating it tosource.mac, which allows downstream IP-based processors to work correctly. The fix usesignore_failure: trueso it is a safe no-op for all log lines that do not carry a MAC prefix.Proposed commit message
Root cause
The dissect_gp processor (tag: dissect_gp, line 254) captures everything before the first literal '(' into source.address, so when Cisco IOS FMANFP prepends an optional L2 MAC address ('00-00-5E-00-53-23 198.51.100.10(56279)'), the token '00-00-5E-00-53-23 198.51.100.10' lands in source.address instead of the bare IP; convert_source_ip then fails because that composite string is not a valid IP literal.
Approach
Insert a grok processor (tag: grok_strip_mac_from_source_address) after the final access-list dissect processor (dissect_gnp, line 288) and before grok_login (line 291). The grok targets source.address with pattern '^%{MAC:source.mac} %{IP:source.address}$' and ignore_failure: true, ignore_missing: true. When Cisco IOS prepends a Layer 2 MAC address before the source IP in FMANFP IPACCESSLOGP log lines, the dissect_gp and dissect_gp_access_list processors capture the combined 'MAC IP' string into source.address; the new grok splits them into source.mac and the bare IP into source.address before convert_source_ip runs. No change to convert_source_ip itself is needed. Add a new pipeline test fixture line in test-cisco-ios.log plus corresponding expected-output entry covering the MAC-prefixed IPACCESSLOGP format.
Implementation
Pipeline changes
Field / mapping changes
—
Sanitized error message
'[value]' is not an IP string literal.Sanitized log (
event_sanitizedexcerpt)<174>15162564: BGP-BETA: 29391555: Jun 26 16:44:58.802 CEST: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list acl_Te0-1-1_in denied tcp 00-00-5E-00-53-23 198.51.100.10(56279) TenGigabitEthernet0/1/1-> 203.0.113.20(45240), 1 packetReviewer concerns
• The new grok processor fires on every event where
source.addressis non-null (not scoped to IPACCESSLOGP/FMANFP events specifically); while harmless due toignore_failure: trueand the specificity of the MAC+IP pattern, it adds a small per-event overhead for all log types that populatesource.address.• If a source address field legitimately starts with something that looks like a Cisco-format MAC followed by an IP in a different log type, the processor would silently overwrite
source.mac; this is an edge case but worth noting.Self-review findings
Self-review invoked: yes (1 cycle)
ifguard matching pipeline convention; addedif: ctx.source?.address != nullFinal validation passed: yes
Risk and classification
Links
b71258f3323b71c4