Skip to content

x-pack/filebeat/input/httpjson: improve/fix logic for health status updates#46332

Merged
efd6 merged 3 commits intoelastic:mainfrom
efd6:httpjson_empty_health
Sep 5, 2025
Merged

x-pack/filebeat/input/httpjson: improve/fix logic for health status updates#46332
efd6 merged 3 commits intoelastic:mainfrom
efd6:httpjson_empty_health

Conversation

@efd6
Copy link
Contributor

@efd6 efd6 commented Sep 1, 2025

Proposed commit message

x-pack/filebeat/input/httpjson: improve/fix logic for health status updates

There was no way to ask the input to not report degraded health in the
case that a template error was required to signal that some aspect of
the input's operation should terminate, outside of the template itself.
This change fixes that.

Also fix buggy cursor template tests that were identified by this
improvement.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

@efd6 efd6 self-assigned this Sep 1, 2025
@efd6 efd6 added enhancement Filebeat Filebeat bugfix Team:Security-Service Integrations Security Service Integrations Team backport-8.18 Automated backport to the 8.18 branch backport-9.0 Automated backport to the 9.0 branch backport-8.19 Automated backport to the 8.19 branch backport-9.1 Automated backport to the 9.1 branch labels Sep 1, 2025
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Sep 1, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2025

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

There was no way to ask the input to not report degraded health in the
case that a template error was required to signal that some aspect of
the input's operation should terminate, outside of the template itself.
This change fixes that.

Also fix buggy cursor template tests that were identified by this
improvement.
@efd6 efd6 force-pushed the httpjson_empty_health branch from 179eea8 to 7656770 Compare September 1, 2025 00:17
@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2025

🔍 Preview links for changed docs

@efd6 efd6 changed the title x-pack/httpjson: improve/fix logic for health status updates Sep 1, 2025
@efd6 efd6 marked this pull request as ready for review September 1, 2025 02:12
@efd6 efd6 requested review from a team as code owners September 1, 2025 02:12
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

Copy link
Contributor

@vishaangelova vishaangelova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs LGTM

@efd6 efd6 merged commit 64617f5 into elastic:main Sep 5, 2025
35 of 38 checks passed
mergify bot pushed a commit that referenced this pull request Sep 5, 2025
…pdates (#46332)

There was no way to ask the input to not report degraded health in the
case that a template error was required to signal that some aspect of
the input's operation should terminate, outside of the template itself.
This change fixes that.

Also fix buggy cursor template tests that were identified by this
improvement.

(cherry picked from commit 64617f5)

# Conflicts:
#	docs/reference/filebeat/filebeat-input-httpjson.md
#	x-pack/filebeat/input/httpjson/cursor.go
#	x-pack/filebeat/input/httpjson/cursor_test.go
#	x-pack/filebeat/input/httpjson/transform_append.go
#	x-pack/filebeat/input/httpjson/transform_set.go
#	x-pack/filebeat/input/httpjson/transform_set_test.go
mergify bot pushed a commit that referenced this pull request Sep 5, 2025
…pdates (#46332)

There was no way to ask the input to not report degraded health in the
case that a template error was required to signal that some aspect of
the input's operation should terminate, outside of the template itself.
This change fixes that.

Also fix buggy cursor template tests that were identified by this
improvement.

(cherry picked from commit 64617f5)

# Conflicts:
#	docs/reference/filebeat/filebeat-input-httpjson.md
mergify bot pushed a commit that referenced this pull request Sep 5, 2025
…pdates (#46332)

There was no way to ask the input to not report degraded health in the
case that a template error was required to signal that some aspect of
the input's operation should terminate, outside of the template itself.
This change fixes that.

Also fix buggy cursor template tests that were identified by this
improvement.

(cherry picked from commit 64617f5)

# Conflicts:
#	x-pack/filebeat/input/httpjson/transform_set_test.go
mergify bot pushed a commit that referenced this pull request Sep 5, 2025
…pdates (#46332)

There was no way to ask the input to not report degraded health in the
case that a template error was required to signal that some aspect of
the input's operation should terminate, outside of the template itself.
This change fixes that.

Also fix buggy cursor template tests that were identified by this
improvement.

(cherry picked from commit 64617f5)
efd6 added a commit that referenced this pull request Sep 5, 2025
…ogic for health status updates (#46399)

* x-pack/filebeat/input/httpjson: improve/fix logic for health status updates (#46332)

There was no way to ask the input to not report degraded health in the
case that a template error was required to signal that some aspect of
the input's operation should terminate, outside of the template itself.
This change fixes that.

Also fix buggy cursor template tests that were identified by this
improvement.

(cherry picked from commit 64617f5)

# Conflicts:
#	docs/reference/filebeat/filebeat-input-httpjson.md

* remove irrelevant changelog entries
* remove markdown docs
* add relevant asciidoc text

---------

Co-authored-by: Dan Kortschak <dan.kortschak@elastic.co>
efd6 added a commit that referenced this pull request Sep 5, 2025
…gic for health status updates (#46401)

* x-pack/filebeat/input/httpjson: improve/fix logic for health status updates (#46332)

There was no way to ask the input to not report degraded health in the
case that a template error was required to signal that some aspect of
the input's operation should terminate, outside of the template itself.
This change fixes that.

Also fix buggy cursor template tests that were identified by this
improvement.

(cherry picked from commit 64617f5)

* remove irrelevant changelog entries

---------

Co-authored-by: Dan Kortschak <dan.kortschak@elastic.co>
efd6 added a commit that referenced this pull request Sep 5, 2025
…gic for health status updates (#46400)

* x-pack/filebeat/input/httpjson: improve/fix logic for health status updates (#46332)

There was no way to ask the input to not report degraded health in the
case that a template error was required to signal that some aspect of
the input's operation should terminate, outside of the template itself.
This change fixes that.

Also fix buggy cursor template tests that were identified by this
improvement.

(cherry picked from commit 64617f5)

# Conflicts:
#	x-pack/filebeat/input/httpjson/transform_set_test.go

* remove irrelevant changelog entries
* fix conflicts

---------

Co-authored-by: Dan Kortschak <dan.kortschak@elastic.co>
@efd6 efd6 removed the backport-8.18 Automated backport to the 8.18 branch label Sep 5, 2025
kcreddy added a commit to elastic/integrations that referenced this pull request Sep 22, 2025
…R) workflow (#15230)

aws: Add transforms to Config and Inspector data streams for extended protections (CDR) workflow.

- Add latest transform to Config and Inspector data streams
to help with Cloud Native Vulnerability Management (CNVM)[1] 
and Cloud Security Posture Management (CSPM)[2] workflows.
- Add ILM policy to AWS Config as it does full sync every interval.
- Update minimum kibana version to "^8.19.0 || ^9.1.0"  to ensure 
necessary permissions for transform[3].
- Re-add 3.14.2 changelog entry as it is overwritten in VPC Flow PR[4].
- Skip system tests for securityhub* data streams to avoid fleet health 
degradation due to empty template values by httpjson. This is fixed in 
8.19.4 and 9.1.4 by beats#45810[5] and beats#46332[6]. This skip can 
be removed when the stack version is upgraded to ones containing the fix. 

[1] https://www.elastic.co/guide/en/security/current/vuln-management-overview.html
[2] https://www.elastic.co/docs/solutions/security/cloud/cloud-security-posture-management
[3] elastic/elasticsearch#128350
[4] #15077
[5] elastic/beats#45810
[6] elastic/beats#46332
tehbooom pushed a commit to tehbooom/integrations that referenced this pull request Nov 19, 2025
…R) workflow (elastic#15230)

aws: Add transforms to Config and Inspector data streams for extended protections (CDR) workflow.

- Add latest transform to Config and Inspector data streams
to help with Cloud Native Vulnerability Management (CNVM)[1] 
and Cloud Security Posture Management (CSPM)[2] workflows.
- Add ILM policy to AWS Config as it does full sync every interval.
- Update minimum kibana version to "^8.19.0 || ^9.1.0"  to ensure 
necessary permissions for transform[3].
- Re-add 3.14.2 changelog entry as it is overwritten in VPC Flow PR[4].
- Skip system tests for securityhub* data streams to avoid fleet health 
degradation due to empty template values by httpjson. This is fixed in 
8.19.4 and 9.1.4 by beats#45810[5] and beats#46332[6]. This skip can 
be removed when the stack version is upgraded to ones containing the fix. 

[1] https://www.elastic.co/guide/en/security/current/vuln-management-overview.html
[2] https://www.elastic.co/docs/solutions/security/cloud/cloud-security-posture-management
[3] elastic/elasticsearch#128350
[4] elastic#15077
[5] elastic/beats#45810
[6] elastic/beats#46332
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-8.19 Automated backport to the 8.19 branch backport-9.0 Automated backport to the 9.0 branch backport-9.1 Automated backport to the 9.1 branch bugfix enhancement Filebeat Filebeat Team:Security-Service Integrations Security Service Integrations Team

4 participants