Skip to content

[8.18](backport #44697) Remove ID from list when Filestream inputs fail to be created#44737

Merged
belimawr merged 2 commits into8.18from
mergify/bp/8.18/pr-44697
Jun 10, 2025
Merged

[8.18](backport #44697) Remove ID from list when Filestream inputs fail to be created#44737
belimawr merged 2 commits into8.18from
mergify/bp/8.18/pr-44697

Conversation

@mergify
Copy link
Contributor

@mergify mergify bot commented Jun 10, 2025

Proposed commit message

Filestream was not removing the ID from inputs that failed to start, that made re-starting the same input with a valid configuration to fail with a duplicated ID error. This affects any configuration mode that can be reloaded, Elastic-Agent being the main use case.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

## Author's Checklist

How to test this PR locally

1. Create the following files

filebeat.yml

filebeat.config:
  inputs:
    enabled: true
    path: inputs.d/*.yml
    reload.enabled: true
    reload.period: 2s

queue.mem:
  flush:
    timeout: 0s

output.console:
  pretty: true

inputs.d/filestream.yml

- type: filestream
  id: t-wing
  paths:
    - /**/**/log.log
    # - /tmp/flog.log

/tmp/flog.log

<90>2 2025-06-09T12:31:10.044Z internalworld-class.net sed 3808 ID961 - We need to connect the primary THX protocol!
<62>2 2025-06-09T12:31:10.044Z investorfunctionalities.name velit 2310 ID769 - The SSL monitor is down, hack the open-source capacitor so we can quantify the PCI system!
<49>1 2025-06-09T12:31:10.044Z nationalmatrix.net rerum 798 ID992 - We need to compress the back-end TCP interface!
<187>1 2025-06-09T12:31:10.044Z humanmesh.org dolores 2059 ID720 - Use the multi-byte TCP protocol, then you can program the wireless array!
<188>2 2025-06-09T12:31:10.044Z districtsolutions.net in 333 ID581 - Use the haptic PNG pixel, then you can input the redundant transmitter!
<17>1 2025-06-09T12:31:10.044Z legacycross-media.net sunt 1957 ID621 - Try to quantify the SDD system, maybe it will quantify the solid state capacitor!
<54>1 2025-06-09T12:31:10.044Z internalmesh.io et 9739 ID142 - Hacking the system won't do anything, we need to parse the primary RAM hard drive!
<161>2 2025-06-09T12:31:10.044Z regionalsexy.net assumenda 9612 ID649 - You can't bypass the firewall without bypassing the back-end AI sensor!
<131>2 2025-06-09T12:31:10.044Z futureembrace.info labore 900 ID122 - I'Ll generate the virtual XSS circuit, that should array the USB sensor!
<6>2 2025-06-09T12:31:10.044Z principalrepurpose.net provident 8513 ID99 - Try to hack the SMS interface, maybe it will back up the bluetooth program!

2. Build and Start Filebeat

go build .
./filebeat

Wait for the log error:

Error creating runner from config: cannot create prospector: error while creating filewatcher multiple ** in \"/**/**/log.log\"

3. Edit inputs.d/filestream.yml so it is valid

Remove the first, invalid, path and uncomment the second one. paths section so it looks like this:

  paths:
    #- /**/**/log.log
    - /tmp/flog.log

Save the file and wait to see the ingested events in the console.

## Related issues
## Use cases
## Screenshots
## Logs


This is an automatic backport of pull request #44697 done by Mergify.

Filestream was not removing the ID from inputs that failed to start,
that made re-starting the same input with a valid configuration to
fail with a duplicated ID error. This affects any configuration mode
that can be reloaded, Elastic-Agent being the main use case.

This commit fixes it by removing the ID from a failed input.
---------

Co-authored-by: Anderson Queiroz <me@andersonq.me>
(cherry picked from commit 017cc4f)

# Conflicts:
#	filebeat/input/filestream/internal/input-logfile/manager.go
#	filebeat/input/filestream/internal/input-logfile/manager_test.go
@mergify mergify bot requested a review from a team as a code owner June 10, 2025 18:49
@mergify mergify bot requested review from khushijain21 and leehinman and removed request for a team June 10, 2025 18:49
@mergify mergify bot added backport conflicts There is a conflict in the backported pull request labels Jun 10, 2025
@mergify
Copy link
Contributor Author

mergify bot commented Jun 10, 2025

Cherry-pick of 017cc4f has failed:

On branch mergify/bp/8.18/pr-44697
Your branch is up to date with 'origin/8.18'.

You are currently cherry-picking commit 017cc4fbc.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   CHANGELOG.next.asciidoc

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   filebeat/input/filestream/internal/input-logfile/manager.go
	both modified:   filebeat/input/filestream/internal/input-logfile/manager_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jun 10, 2025
@github-actions github-actions bot added Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team bugfix labels Jun 10, 2025
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jun 10, 2025
@elasticmachine
Copy link
Contributor

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@belimawr belimawr enabled auto-merge (squash) June 10, 2025 20:59
@belimawr belimawr merged commit 1abd1d2 into 8.18 Jun 10, 2025
33 of 36 checks passed
@belimawr belimawr deleted the mergify/bp/8.18/pr-44697 branch June 10, 2025 22:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport bugfix conflicts There is a conflict in the backported pull request Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team

2 participants