[libbeat/output/kafka]: fix panic on Publish after Close

[AndersonQ]

Proposed commit message

A race condition during client shutdown could cause a panic.

If a `Publish` call was in-flight while the client was closing, the underlying Kafka producer's input channel could be closed before the call finished sending all messages in the batch. The subsequent attempt to send the remaining messages on this closed channel would result in a panic.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

• n/a

How to test this PR locally

• try the steps described in the bug Filebeat panics when attempting to shutdown while using Kafka output #46109: (they did not work for me)

• copy the new test, run it on main, you'll see the panic

Related issues

• Closes Filebeat panics when attempting to shutdown while using Kafka output #46109

[github-actions]

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @AndersonQ? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 46109-kafka-output-panic upstream/46109-kafka-output-panic
git merge upstream/main
git push upstream 46109-kafka-output-panic

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 46109-kafka-output-panic upstream/46109-kafka-output-panic
git merge upstream/main
git push upstream 46109-kafka-output-panic

[Copilot]

Pull Request Overview

Fixes a race condition in the Kafka output client that could cause a panic during shutdown when Publish calls were in-flight while the client was closing.

• Adds proper channel close detection in the Publish method to prevent sending on closed channels
• Implements graceful event dropping with logging when the client is shutting down
• Adds a comprehensive test to reproduce and verify the fix for the race condition

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
libbeat/outputs/kafka/client.go	Adds select statement to handle closed channel during publish operations
libbeat/outputs/kafka/client_test.go	Adds test case to reproduce the shutdown panic scenario
CHANGELOG.next.asciidoc	Documents the bug fix in the changelog

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[Copilot]

Pull Request Overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 46109-kafka-output-panic upstream/46109-kafka-output-panic
git merge upstream/main
git push upstream 46109-kafka-output-panic

[andrzej-stencel]

Thanks!

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 46109-kafka-output-panic upstream/46109-kafka-output-panic
git merge upstream/main
git push upstream 46109-kafka-output-panic

[github-actions]

@Mergifyio backport 8.18 8.19 9.0 9.1

[mergify]

backport 8.18 8.19 9.0 9.1

✅ Backports have been created

Details

• #46709 [8.18](backport #46446) [libbeat/output/kafka]: fix panic on Publish after Close has been created for branch 8.18
• #46710 [8.19](backport #46446) [libbeat/output/kafka]: fix panic on Publish after Close has been created for branch 8.19
• #46711 [9.0](backport #46446) [libbeat/output/kafka]: fix panic on Publish after Close has been created for branch 9.0
• #46712 [9.1](backport #46446) [libbeat/output/kafka]: fix panic on Publish after Close has been created for branch 9.1