kafka_log: clarify default protocol version in description

[mauri870]

Proposed commit message

The default Kafka protocol version depends on the filebeat version. For 8.x it defaults to 1.0.0, and for 9.0+ it defaults to 2.1.0. Update the description to clarify this.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 7f5588c

History

• 💔 Build #35413 failed 62a53d2

cc @mauri870

[lalit-satapathy]

@agithomas / @stefans-elastic for review, given recent kafka upgrades.

[agithomas]

The change looks good to me. My suggestion would be to add a troubleshooting section in the README document to help those users migrating from the 8.x+ stack to the 9.x+ stack. In that process, the users would fail to notice this dependency.

@stefans-elastic , as you have worked on the sarama library dependency issue pertaining to the metrics, kindly check from your end as well. Thanks.

[stefans-elastic]

I agree: the change looks good. Also it matches the version we use in metricbeat.
Overall according to sarama code comment it is better to use lower version in sarama config than actual Kafka version it is running against because otherwise it may lead to random breakages.

[agithomas]

Code owner approval based on the above-mentioned validations.

[elastic-vault-github-plugin-prod]

Package kafka_log - 1.9.1 containing this change is available at https://epr.elastic.co/package/kafka_log/1.9.1/