[panw] Update event.created field to follow ECS spec - #10731
Merged
Conversation
The ECS spec states `event.created` should be the time the event is first seen by the agent or ingested. The panw integration was not following this, and was instead setting it to the PANOS event timestamp. This corrects the integration to follow ECS properly. `event.created` is now set by the filebeat @timestamp, which is the time the agent first sees the event. The PANOS timestamp is still available in the `panw.panos.received_time` field.
🚀 Benchmarks reportTo see the full report comment with |
andrewkroh
added
Integration:panw
|
Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices) |
mjwolf
force-pushed
the
panw-event-created
branch
2 times, most recently
from
August 8, 2024 21:33
65d6e81 to
96aed05
Compare
* Don't set event.created field, this is added by filebeat when it receives the event. * Set '@timestamp' to be set from received_time, if high_resolution_timestamp is not available.
mjwolf
force-pushed
the
panw-event-created
branch
from
August 8, 2024 21:39
96aed05 to
5c32971
Compare
| "category": [ | ||
| "configuration" | ||
| ], | ||
| "created": "2021-10-25T20:25:39.000-04:00", |
Contributor
There was a problem hiding this comment.
We're losing event.created here. I think we need to copy the initial @timestamp to event.created at the beginning of the pipeline to fix this.
Set event.created in one of the first filebeat processors, before syslog processor. `event.created` needs to be set in filebeat, since it's defined as the time filebeat/agent first processes the event. It also needs to be set before the syslog processor, since syslog will set the timestamp to the log message's timestamp.
taylor-swanson
approved these changes
Aug 15, 2024
mjwolf
enabled auto-merge (squash)
August 15, 2024 20:23
💚 Build Succeeded
History
cc @mjwolf |
|
|
Package panw - 4.0.0 containing this change is available at https://epr.elastic.co/search?package=panw |
harnish-crest-data
pushed a commit
to chavdaharnish/integrations
that referenced
this pull request
Feb 4, 2025
The [ECS spec states](https://www.elastic.co/guide/en/ecs/current/ecs-event.html#field-event-created) `event.created` should be the time the event is first seen by the agent or ingested. The panw integration was not following this and was instead setting it to the PANOS event timestamp. This corrects the integration so that it follows ECS properly. These field changes have been made: * `event.created` is set from filebeat's initial timestamp (before modification by the syslog processor or ingest pipeline). * `panw.panos.received_time` is now defined as the PANOS log timestamp (it is the same value that was previous in `event.created`) * `panw.panos.generated_time` is added to hold the PANOS generated time * `@timestamp` now holds the `panw.panos.high_resolution_timestamp` value, or if it isn't available `panw.panos.received_time` This is also a major version upgrade, because of these changed field definitions.
harnish-crest-data
pushed a commit
to chavdaharnish/integrations
that referenced
this pull request
Feb 5, 2025
The [ECS spec states](https://www.elastic.co/guide/en/ecs/current/ecs-event.html#field-event-created) `event.created` should be the time the event is first seen by the agent or ingested. The panw integration was not following this and was instead setting it to the PANOS event timestamp. This corrects the integration so that it follows ECS properly. These field changes have been made: * `event.created` is set from filebeat's initial timestamp (before modification by the syslog processor or ingest pipeline). * `panw.panos.received_time` is now defined as the PANOS log timestamp (it is the same value that was previous in `event.created`) * `panw.panos.generated_time` is added to hold the PANOS generated time * `@timestamp` now holds the `panw.panos.high_resolution_timestamp` value, or if it isn't available `panw.panos.received_time` This is also a major version upgrade, because of these changed field definitions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed commit message
The ECS spec states
event.createdshould be the time the event is first seen by the agent or ingested. The panw integration was not following this, and was instead setting it to the PANOS event timestamp.This corrects the integration to follow ECS properly.
These field changes have been made:
event.createdis set from filebeat's initial timestamp (before modification by the syslog processor or ingest pipeline).panw.panos.received_timeis now defined as the PANOS log timestamp (it is the same value that was previous inevent.created)panw.panos.generated_timeis added to hold the PANOS generated time@timestampnow holds thepanw.panos.high_resolution_timestampvalue, or if it isn't availablepanw.panos.received_timeThis is also a major version upgrade, because of these changed field definitions.
Checklist
changelog.ymlfile.