Skip to content

[GCP] Add GCP Compute Metricset - #2301

Merged
endorama merged 7 commits into
elastic:gcp-metricsfrom
legoguy1000:490-gcp-compute
Dec 31, 2021
Merged

[GCP] Add GCP Compute Metricset#2301
endorama merged 7 commits into
elastic:gcp-metricsfrom
legoguy1000:490-gcp-compute

Conversation

@legoguy1000

@legoguy1000 legoguy1000 commented Dec 2, 2021

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds GCP Compute Metricset

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • If I'm introducing a new feature, I have modified the Kibana version constraint in my package's manifest.yml file to point to the latest Elastic stack release (e.g. ^7.13.0).

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@elasticmachine

elasticmachine commented Dec 2, 2021

Copy link
Copy Markdown

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-12-29T21:30:53.845+0000

  • Duration: 18 min 21 sec

  • Commit: 32e57ee

Test stats 🧪

Test Results
Failed 0
Passed 43
Skipped 0
Total 43

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.
@legoguy1000
legoguy1000 marked this pull request as ready for review December 2, 2021 04:29
@legoguy1000

Copy link
Copy Markdown
Contributor Author

@endorama next one ready for your review

@legoguy1000

Copy link
Copy Markdown
Contributor Author

@endorama Any thoughts on this one?

@kaiyan-sheng kaiyan-sheng added the Team:Integrations Label for the Integrations team label Dec 14, 2021
@elasticmachine

Copy link
Copy Markdown

Pinging @elastic/integrations (Team:Integrations)

@kaiyan-sheng kaiyan-sheng left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Waiting for @endorama 's review 🙂

@kaiyan-sheng

Copy link
Copy Markdown

/test

Comment thread packages/gcp/data_stream/compute/agent/stream/stream.yml.hbs Outdated
Comment thread packages/gcp/data_stream/compute/fields/agent.yml Outdated
Comment thread packages/gcp/docs/compute.md Outdated
@endorama

Copy link
Copy Markdown
Contributor

Sorry for the delay, I've been stuck adding tests to the billing data stream. I did a first round of review and noticed something strange with field names.

@legoguy1000

Copy link
Copy Markdown
Contributor Author

@endorama updates pushed

@endorama

endorama commented Dec 20, 2021

Copy link
Copy Markdown
Contributor

@legoguy1000 thanks! May you please check dashboards too? It seems those still contains old values.

@legoguy1000

Copy link
Copy Markdown
Contributor Author

@endorama Dashboards should be updated.

@endorama endorama left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the updates! I see tests are still failing, but wasn't able to pin point the exact issue.

@endorama

Copy link
Copy Markdown
Contributor

rerunning tests in case has been a transactional issue.

/test

@legoguy1000

Copy link
Copy Markdown
Contributor Author

@endorama It looks to be a mismatch between the sample _event.json and the defined fields for the compute dataset. The sample events from Metricbeat, https://github.com/elastic/beats/tree/master/x-pack/metricbeat/module/gcp/compute/_meta, were not updated for the new field names and as such don't match the fields.yml. Do you happen to have updated sample events that could be used?

@endorama

Copy link
Copy Markdown
Contributor

@legoguy1000 I'm working on updating the sample data, thanks for the hint

@endorama

Copy link
Copy Markdown
Contributor

I've been able to update the sample data and working on a PR in beats to update them.

@legoguy1000

Copy link
Copy Markdown
Contributor Author

I've been able to update the sample data and working on a PR in beats to update them.

Cool, feel free to push that update to this branch or send it to me and I'll update it.

@endorama

Copy link
Copy Markdown
Contributor

@legoguy1000 I started working on updating the sample events in beats in the linked PR, but is taking longer than expected. In any case I've been able to update the compute metricset sample events, so you may take those to update this PR, and I expect that to work. Meanwhile I'll work on updating the beats side too.

@legoguy1000

Copy link
Copy Markdown
Contributor Author

@endorama looks like it worked.

@endorama

Copy link
Copy Markdown
Contributor

Thank you! I'm merging this as is good to go, I'll open a separate PR with testing for this data stream.

@legoguy1000 next week I'll be off, so feel free to open the next PR but I won't be able to look at it before the 10th of January. Thank you for the contribution and happy new year!

@endorama
endorama merged commit 7567409 into elastic:gcp-metrics Dec 31, 2021
@legoguy1000
legoguy1000 deleted the 490-gcp-compute branch December 31, 2021 16:26
endorama pushed a commit that referenced this pull request Feb 1, 2022
endorama pushed a commit that referenced this pull request Mar 17, 2022
endorama pushed a commit that referenced this pull request Mar 22, 2022
This commit has been changed to remove changes to README included in
 #2141 to allow documentation changes from #2842
ghost pushed a commit that referenced this pull request Mar 30, 2022
* gcp: stub logs and metrics doc files

* [GCP] Add GCP Billing Metricset (#2141)

* [GCP] Add GCP Compute Metricset (#2301)

* [GCP] Add GCP Billing Metricset (#2141)

* [GCP] Add GCP Compute Metricset (#2301)

* Add firestore data_stream

* Fix formatting issue

* Add gcp tests env vars in Jenkinsfile

* Add missing env fields

* Update terraform deployer

* Update system test

* Fix credentials json field

* Format & add period

* Update docs

* Update default period for firestoer

* Update terraform deployer

* Remove period from system test

* Update manifest version

* Update manifest file

* Add default project id

* Fix owner

* Update audit docs

* Fix terraform deployer

* Remove redundant alternative host

* Fix project variable

* Remove default project string

* Remove firestore datastream tests

* Revert "Add gcp tests env vars in Jenkinsfile"

This reverts commit ceea735.

Co-authored-by: Edoardo Tenani <edoardo.tenani@elastic.co>
Co-authored-by: Alex Resnick <adr8292@gmail.com>
endorama pushed a commit that referenced this pull request Apr 5, 2022
This commit has been changed to remove changes to README included in
 #2141 to allow documentation changes from #2842
endorama pushed a commit that referenced this pull request Jun 9, 2022
This commit has been changed to remove changes to README included in
 #2141 to allow documentation changes from #2842
endorama pushed a commit that referenced this pull request Jul 12, 2022
This commit has been changed to remove changes to README included in
 #2141 to allow documentation changes from #2842
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Team:Integrations Label for the Integrations team

4 participants