Skip to content

fix: Use KafkaWriter in dataobj-tee - #21732

Merged
grobinson-grafana merged 1 commit into
mainfrom
grobinson/use-kafka-writer-dataobj-tee
Apr 30, 2026
Merged

fix: Use KafkaWriter in dataobj-tee#21732
grobinson-grafana merged 1 commit into
mainfrom
grobinson/use-kafka-writer-dataobj-tee

Conversation

@grobinson-grafana

@grobinson-grafana grobinson-grafana commented Apr 29, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

This commit uses the KafkaWriter in the dataobj-tee so both topics use the same limits in pkg/kafka/client/writer_client.go.

In future, we will deprecate the limits from writer_client.go and use MaxBufferedBytes instead, since twmb/franz-go/issues/777 has been fixed.

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • Documentation added
  • Tests updated
  • Title matches the required conventional commits format, see here
  • Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
  • If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

Note

Medium Risk
Touches the distributor write path to Kafka by changing the producer type and ProduceSync call signature; behavior should be equivalent but miswiring could break or alter tee writes under load.

Overview
The data-object tee now writes via the distributor’s shared KafkaProducer (kafkaWriter) instead of using a dedicated *kgo.Client, so both Kafka topics follow the same writer limits/configuration.

This updates DataObjTee/NewDataObjTee to accept KafkaProducer and adjusts the ProduceSync invocation to match the producer API.

Reviewed by Cursor Bugbot for commit 6dfdc5c. Bugbot is set up for automated code reviews on this repo. Configure here.

This commit uses the KafkaWriter in the dataobj-tee so both topics
use the same limits in pkg/kafka/client/writer_client.go.

In future, we will deprecate the limits from writer_client.go
and use MaxBufferedBytes instead, since twmb/franz-go/issues/777
has been fixed.
@grobinson-grafana
grobinson-grafana requested a review from a team as a code owner April 29, 2026 14:59
@grobinson-grafana
grobinson-grafana merged commit d8f4d0f into main Apr 30, 2026
90 of 91 checks passed
@grobinson-grafana
grobinson-grafana deleted the grobinson/use-kafka-writer-dataobj-tee branch April 30, 2026 13:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2 participants