Skip to content

[GCP] [DNS logs] Don't parse rdata when it is empty - #8447

Merged
zmoog merged 6 commits into
elastic:mainfrom
zmoog:zmoog/fix-empty-rdata-in-gcp-dns-logs
Nov 21, 2023
Merged

[GCP] [DNS logs] Don't parse rdata when it is empty#8447
zmoog merged 6 commits into
elastic:mainfrom
zmoog:zmoog/fix-empty-rdata-in-gcp-dns-logs

Conversation

@zmoog

@zmoog zmoog commented Nov 9, 2023

Copy link
Copy Markdown
Contributor

Proposed commit message

Extends the pipeline script processor check to process rdata field only if it's not null and not an empty string.

A user reported pipeline errors occurring when the rdata is an empty string.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

cd packages/gcp
$ elastic-package test pipeline -v  

2023/11/09 13:59:01 DEBUG GET https://127.0.0.1:5601/api/status
--- Test results for package: gcp - START ---
╭─────────┬────────────────────┬───────────┬──────────────────────────┬────────┬──────────────╮
│ PACKAGE │ DATA STREAM        │ TEST TYPE │ TEST NAME                │ RESULT │ TIME ELAPSED │
├─────────┼────────────────────┼───────────┼──────────────────────────┼────────┼──────────────┤
│ gcp     │ audit              │ pipeline  │ test-audit.log           │ PASS   │  15.953834ms │
│ gcp     │ audit              │ pipeline  │ test-sdh-3695.log        │ PASS   │   2.896625ms │
│ gcp     │ compute            │ pipeline  │ test-gcp-compute.json    │ PASS   │      1.758ms │
│ gcp     │ dns                │ pipeline  │ test-dns-empty-rdata.log │ PASS   │   4.668583ms │
│ gcp     │ dns                │ pipeline  │ test-dns.log             │ PASS   │  11.604458ms │
│ gcp     │ firewall           │ pipeline  │ test-firewall.log        │ PASS   │   7.663833ms │
│ gcp     │ loadbalancing_logs │ pipeline  │ test-load-balancer.log   │ PASS   │   3.258583ms │
│ gcp     │ vpcflow            │ pipeline  │ test-vpcflow.log         │ PASS   │  60.242458ms │
╰─────────┴────────────────────┴───────────┴──────────────────────────┴────────┴──────────────╯
--- Test results for package: gcp - END   ---
Done
@zmoog zmoog self-assigned this Nov 9, 2023
@zmoog zmoog added Team:Cloud-Monitoring Label for the Cloud Monitoring team bug Something isn't working, use only for issues labels Nov 9, 2023
@zmoog
zmoog marked this pull request as ready for review November 9, 2023 13:03
@zmoog
zmoog requested review from a team as code owners November 9, 2023 13:03
@elasticmachine

elasticmachine commented Nov 9, 2023

Copy link
Copy Markdown

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-11-21T04:57:06.579+0000

  • Duration: 19 min 57 sec

Test stats 🧪

Test Results
Failed 0
Passed 69
Skipped 0
Total 69

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine

elasticmachine commented Nov 9, 2023

Copy link
Copy Markdown

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (6/6) 💚
Files 100.0% (6/6) 💚
Classes 100.0% (6/6) 💚
Methods 87.826% (101/115) 👎 -4.274
Lines 95.124% (1502/1579) 👍 3.559
Conditionals 100.0% (0/0) 💚
Comment thread packages/gcp/data_stream/dns/_dev/test/pipeline/test-dns-empty-rdata.log Outdated
Comment thread packages/gcp/data_stream/dns/elasticsearch/ingest_pipeline/default.yml Outdated
@zmoog
zmoog force-pushed the zmoog/fix-empty-rdata-in-gcp-dns-logs branch from 9bf8fea to 7a4be8e Compare November 14, 2023 12:49
@zmoog
zmoog requested a review from efd6 November 16, 2023 18:02

@kaiyan-sheng kaiyan-sheng left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@zmoog

zmoog commented Nov 17, 2023

Copy link
Copy Markdown
Contributor Author

Hey @efd6, do you think we should make other changes to this PR, or we're good?

zmoog and others added 6 commits November 21, 2023 05:53
A user reported pipeline errors occurring when the `rdata` is set to
an empty string.

I am looking for the actual sample document from GCP, but in the
meantime, I'm trying to reproduce the issue by modifying another test
document.
…-rdata.log

Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>
…ault.yml

Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>
@zmoog
zmoog force-pushed the zmoog/fix-empty-rdata-in-gcp-dns-logs branch from 7a4be8e to 09504e2 Compare November 21, 2023 04:56
@zmoog
zmoog merged commit 536fd1e into elastic:main Nov 21, 2023
@zmoog
zmoog deleted the zmoog/fix-empty-rdata-in-gcp-dns-logs branch November 21, 2023 16:02
@elasticmachine

Copy link
Copy Markdown

Package gcp - 2.31.2 containing this change is available at https://epr.elastic.co/search?package=gcp

@andrewkroh andrewkroh added the Integration:gcp Google Cloud Platform label Nov 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working, use only for issues Integration:gcp Google Cloud Platform Team:Cloud-Monitoring Label for the Cloud Monitoring team

5 participants