[Contextual Security][GCP auditlogs] add type-specific entity fields to gcp-auditlogs data stream - #15930
Conversation
fix and add gcp-aduitlogs ingest pipeline tests
alexreal1314
added
documentation
andrewkroh
added
the
Team:Security-Service Integrations
|
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations) |
chrisberkhout
left a comment
chrisberkhout
left a comment
There was a problem hiding this comment.
Nice. One minor point in the Painless.
| return; | ||
| } | ||
| String[] parts = path.splitOnToken('.'); | ||
| def current = ctx; |
There was a problem hiding this comment.
This assignment is redundant. The function parameter could be named current and could be reassigned in the body without affecting ctx at the call site.
albertoblaz
left a comment
albertoblaz
left a comment
There was a problem hiding this comment.
Had a quick look and looks good overall. The changes in the Painless script make sense and I saw the support for the new fields while deprecating the old ones.
I just don't dare to accept since I have very little context on specific implementation details
🚀 Benchmarks reportPackage
|
| Data stream | Previous EPS | New EPS | Diff (%) | Result |
|---|---|---|---|---|
audit |
0 | 1173.71 | 1173.71 ( - %) | 👍 |
compute |
0 | 14285.71 | 14285.71 ( - %) | 👍 |
dns |
0 | 2304.15 | 2304.15 ( - %) | 👍 |
firewall |
0 | 4405.29 | 4405.29 ( - %) | 👍 |
loadbalancing_logs |
0 | 3215.43 | 3215.43 ( - %) | 👍 |
vpcflow |
0 | 2976.19 | 2976.19 ( - %) | 👍 |
3db4393 to
fef5900
Compare
fef5900 to
ee3a470
Compare
|
/test benchmark fullreport |
💚 Build Succeeded
History
|
alexreal1314
changed the title
albertoblaz
left a comment
albertoblaz
left a comment
There was a problem hiding this comment.
I've got zero experience with Painless but the whole script and mappings make sense to me
|
Package gcp - 2.46.0 containing this change is available at https://epr.elastic.co/package/gcp/2.46.0/ |
| @@ -219,6 +219,126 @@ processors: | |||
| field: target.entity.id | |||
| value: "{{{json.resource.disk_id}}}" | |||
| if: ctx.json?.resouce?.disk_id != null | |||
There was a problem hiding this comment.
@alexreal1314 is it a typo? resouce and not resource
6 participants
Proposed commit message
This PR enhances entity classification by populating type-specific fields - user.entity.id, service.entity.id, host.entity.id, entity.id for actor fields alongside existing legacy fields.
Same is relevant for target fields - user.target.entity.id, service.target.entity.id, host.target.entity.id, entity.target.id.
The implementation uses ARN parsing and resource ID pattern matching to classify AWS entities into appropriate categories, enabling better entity visualization in Security Solution while maintaining backward compatibility.
WHAT:
Ingest Pipeline Updates (default.yml)
Implementation:
WHY:
Primary Goals
Checklist
changelog.ymlfile.Author's Checklist
How to test this PR locally
logs-*and apply filterdata_stream.dataset: gcp.auditRelated issues
Screenshots
GCP auditlogs:
Every document with actor.entity.id field is mapped to at least one of user/host/service/''.entity.id:
query:
Every document with target.entity.id field is mapped to at least one of user/host/service.target.entity.id or entity.target.id:
query:
classifications test:
query:
results: