Skip to content

[Azure Application Gateway] Fix parsing error client port is blank and adjust for timeStamp - #5073

Merged
zmoog merged 11 commits into
elastic:mainfrom
nicpenning:patch-2
Feb 3, 2023
Merged

[Azure Application Gateway] Fix parsing error client port is blank and adjust for timeStamp#5073
zmoog merged 11 commits into
elastic:mainfrom
nicpenning:patch-2

Conversation

@nicpenning

Copy link
Copy Markdown
Contributor

Pipeline fails to parse Azure Application Gateway logs when the client port field exists with no data.

error.message : For input string: \"\" convert

What does this PR do?

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
Pipeline fails to parse Azure Application Gateway logs when the client port field exists with no data.

```
error.message : For input string: \"\" convert
```
@nicpenning
nicpenning requested a review from a team as a code owner January 20, 2023 21:17
@elasticmachine

elasticmachine commented Jan 20, 2023

Copy link
Copy Markdown

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-02-03T15:55:17.758+0000

  • Duration: 15 min 46 sec

Test stats 🧪

Test Results
Failed 0
Passed 124
Skipped 0
Total 124

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@nicpenning nicpenning changed the title Fix parsing error when no client port exists Jan 21, 2023
@nicpenning nicpenning changed the title [azure-application-gateway] Fix parsing error when no client port exists Jan 21, 2023
@zmoog

zmoog commented Jan 24, 2023

Copy link
Copy Markdown
Contributor

Hey @nicpenning, thank you for contributing to the Azure Application Gateway integration with this fix!

@zmoog
zmoog self-requested a review January 24, 2023 11:25
@zmoog zmoog added bug Something isn't working, use only for issues Team:Cloud-Monitoring Label for the Cloud Monitoring team labels Jan 24, 2023
@zmoog zmoog self-assigned this Jan 24, 2023
The fields are not in the same order but is very similar to other examples except with the blank clientPort field.
For some reason, the _ingest.timestamp is outside of the _source field. I have ommited it from here.

Furthermore, it seems timestamp changed to timeStamp, so the pipeline will need to be updated.
@nicpenning nicpenning changed the title [Azure Application Gateway] Fix parsing error when no client port exists Jan 24, 2023
@nicpenning

nicpenning commented Jan 25, 2023

Copy link
Copy Markdown
Contributor Author

When I was going through and trying to provide some example data I stumbled upon another issue which was the timestamp in the document I had was using timeStamp so I added that to the pipeline.

@nicpenning
nicpenning requested a review from zmoog January 27, 2023 13:10
@nicpenning

Copy link
Copy Markdown
Contributor Author

Not sure where my comment went, but yes, this data is from the Azure Gov cloud. 😀

@nicpenning

Copy link
Copy Markdown
Contributor Author

Okay @zmoog, how is everything looking now?

@zmoog

zmoog commented Jan 31, 2023

Copy link
Copy Markdown
Contributor

/test

@elasticmachine

elasticmachine commented Jan 31, 2023

Copy link
Copy Markdown

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (10/10) 💚
Files 86.364% (19/22) 👎 -13.636
Classes 86.364% (19/22) 👎 -13.636
Methods 83.422% (156/187) 👎 -16.578
Lines 85.07% (2792/3282) 👍 14.272
Conditionals 100.0% (0/0) 💚
@nicpenning

Copy link
Copy Markdown
Contributor Author

Any further action needed for this? I don't want this to get too stale.

@zmoog

zmoog commented Feb 3, 2023

Copy link
Copy Markdown
Contributor

Any further action needed for this? I don't want this to get too stale.

Yep, someone must check and fix the errors in the CI pipeline. I can't merge the PR if they aren't all green.

Let me take a look at the failing one.

@zmoog

zmoog commented Feb 3, 2023

Copy link
Copy Markdown
Contributor

/test

The pipeline test was failing due to differences between the expected
and actual document.

I fixed the pipeline tests error by regenerating the expected document
using the following command:

    elastic-package test pipeline --generate

The `--generate` option runs the pipeline and save the result in a
test result file.
@zmoog
zmoog merged commit 30fb89f into elastic:main Feb 3, 2023
@zmoog

zmoog commented Feb 3, 2023

Copy link
Copy Markdown
Contributor

@nicpenning, the bot will comment on this PR when the new version is available.

Thank you for your contribution and your patience in these back and forth!

@nicpenning
nicpenning deleted the patch-2 branch February 3, 2023 16:32
@elasticmachine

Copy link
Copy Markdown

Package azure - 1.5.6 containing this change is available at https://epr.elastic.co/search?package=azure

orestisfl pushed a commit to orestisfl/integrations that referenced this pull request May 15, 2026
…d adjust for timeStamp (elastic#5073)

* Fix parsing error when no client port exists

Pipeline fails to parse Azure Application Gateway logs when the client port field exists with no data.

```
error.message : For input string: \"\" convert
```

* Update changelog.yml

* Update manifest.yml

* Update test-application-gateway-raw.log

The fields are not in the same order but is very similar to other examples except with the blank clientPort field.

* Update test-application-gateway-raw.log-expected.json

For some reason, the _ingest.timestamp is outside of the _source field. I have ommited it from here.

Furthermore, it seems timestamp changed to timeStamp, so the pipeline will need to be updated.

* Add new timeStamp field

* Update changelog.yml

* Change to lowercase timeStamp instead

* Change the lowercase to rename

* Correct field rename

* Regenerate the expected test document

The pipeline test was failing due to differences between the expected
and actual document.

I fixed the pipeline tests error by regenerating the expected document
using the following command:

    elastic-package test pipeline --generate

The `--generate` option runs the pipeline and save the result in a
test result file.

---------

Co-authored-by: Maurizio Branca <maurizio.branca@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working, use only for issues Team:Cloud-Monitoring Label for the Cloud Monitoring team

3 participants