Skip to content

add missing fields gcp audit logs - #10886

Merged
haetamoudi merged 8 commits into
elastic:mainfrom
haetamoudi:enhance-gcp-audit-integration
Sep 16, 2024
Merged

add missing fields gcp audit logs#10886
haetamoudi merged 8 commits into
elastic:mainfrom
haetamoudi:enhance-gcp-audit-integration

Conversation

@haetamoudi

@haetamoudi haetamoudi commented Aug 26, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

Proposed commit message

Add policy_violation_info, metadata and related fields to GCP audit logs.

Changes

  • Add policy_violation_info, metadata and related fields to audit logs.
  • Add more audit logs for testing
  • Update GCP audit log dashboard to use correct email field

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • Sample logs do not contain sensitive data
  • Change in dashboard query won't break the visualization

How to test this PR locally

Follow the public documentation to ingest GCP Audit logs https://www.elastic.co/docs/current/integrations/gcp
@haetamoudi haetamoudi added the enhancement New feature or request label Aug 26, 2024
@andrewkroh andrewkroh added the Integration:gcp Google Cloud Platform label Aug 26, 2024
@haetamoudi haetamoudi mentioned this pull request Aug 26, 2024
Closed
8 tasks
@haetamoudi
haetamoudi marked this pull request as ready for review August 26, 2024 15:34
@haetamoudi
haetamoudi requested review from a team as code owners August 26, 2024 15:34
@andrewkroh andrewkroh added the Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] label Aug 26, 2024
@elasticmachine

elasticmachine commented Aug 26, 2024

Copy link
Copy Markdown

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)
@elasticmachine

elasticmachine commented Aug 26, 2024

Copy link
Copy Markdown

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport
@ebeahan

ebeahan commented Sep 5, 2024 •
edited
Loading

Copy link
Copy Markdown
Member

@efd6 @ShourieG would one of you be able to review the changes to the audit data stream for @haetamoudi here?

@kgeller we can't give it a ✅ since we're not codeowners but will you look over too for any feedback?

EDIT: also the coverage failure can be ignored. @haetamoudi actually looking at some improvements in elastic/elastic-package#2063.
efd6
efd6 reviewed Sep 5, 2024
View reviewed changes
Comment thread packages/gcp/changelog.yml Outdated
Comment thread packages/gcp/data_stream/audit/_dev/test/pipeline/test-audit.log Outdated
Comment thread packages/gcp/data_stream/audit/_dev/test/pipeline/test-audit.log Outdated
Comment thread packages/gcp/data_stream/audit/elasticsearch/ingest_pipeline/default.yml Outdated
Comment thread packages/gcp/data_stream/audit/elasticsearch/ingest_pipeline/default.yml Outdated
kgeller
kgeller reviewed Sep 5, 2024
View reviewed changes

@kgeller kgeller left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks great! Just a couple of suggestions 😄
Comment thread packages/gcp/changelog.yml Outdated
Comment thread packages/gcp/data_stream/audit/elasticsearch/ingest_pipeline/default.yml
Comment thread packages/gcp/data_stream/audit/elasticsearch/ingest_pipeline/default.yml
haetamoudi and others added 6 commits September 6, 2024 09:19
@haetamoudi @efd6
Update packages/gcp/data_stream/audit/_dev/test/pipeline/test-audit.log
d25b938 
Co-authored-by: Dan Kortschak <dan.kortschak@elastic.co>
@haetamoudi @efd6
Update packages/gcp/changelog.yml
5e93e62 
Co-authored-by: Dan Kortschak <dan.kortschak@elastic.co>
@haetamoudi @efd6
Update packages/gcp/data_stream/audit/elasticsearch/ingest_pipeline/d…
e5f6761 
…efault.yml

Co-authored-by: Dan Kortschak <dan.kortschak@elastic.co>
@haetamoudi @efd6
Update packages/gcp/data_stream/audit/elasticsearch/ingest_pipeline/d…
e909a3d 
…efault.yml

Co-authored-by: Dan Kortschak <dan.kortschak@elastic.co>
@haetamoudi
allow duplicate false in gcp audit log
0c20551
@haetamoudi
Merge branch 'main' into enhance-gcp-audit-integration
1dfd370
@elasticmachine

elasticmachine commented Sep 6, 2024

Copy link
Copy Markdown

💚 Build Succeeded

History
@elastic-sonarqube

elastic-sonarqube Bot commented Sep 6, 2024

Copy link
Copy Markdown

Quality Gate failed Quality Gate failed

Failed conditions
75.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube
@haetamoudi
haetamoudi requested review from efd6 and kgeller September 6, 2024 18:23
@haetamoudi

haetamoudi commented Sep 10, 2024

Copy link
Copy Markdown
Contributor Author

@efd6 I addressed the changes from the comments, let me know if anything if missing to get approval
@haetamoudi
haetamoudi merged commit 8130976 into elastic:main Sep 16, 2024
@haetamoudi
haetamoudi deleted the enhance-gcp-audit-integration branch September 16, 2024 08:40
@elasticmachine

elasticmachine commented Sep 16, 2024

Copy link
Copy Markdown

Package gcp - 2.38.0 containing this change is available at https://epr.elastic.co/search?package=gcp
harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 4, 2025
@haetamoudi
[gcp] Add policy_violation_info metadata and related fields to GCP au…
796660d 
…dit logs (elastic#10886)

* add missing fields gcp audit logs

* update changelog entry
harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 5, 2025
@haetamoudi
[gcp] Add policy_violation_info metadata and related fields to GCP au…
591216f 
…dit logs (elastic#10886)

* add missing fields gcp audit logs

* update changelog entry
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Integration:gcp Google Cloud Platform Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

6 participants

@haetamoudi @elasticmachine @ebeahan @kgeller @efd6 @andrewkroh