Skip to content

refactor(ingestor): WAL Replay controller: Use singleflight to ensure only one inflight flush is happening - #19831

Merged
Segflow merged 8 commits into
mainfrom
refactor-ingester-replay-controll-use-singleflight
Nov 12, 2025
Merged

refactor(ingestor): WAL Replay controller: Use singleflight to ensure only one inflight flush is happening#19831
Segflow merged 8 commits into
mainfrom
refactor-ingester-replay-controll-use-singleflight

Conversation

@Segflow

@Segflow Segflow commented Nov 11, 2025

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:
When going through the codebase to understand how we replaying the WAL etc.. I come across this interesting piece of code and I thought I could simplify it by using the singleflight package instead of current approach (complex sync.Mutex and sync.Cond)

Special notes for your reviewer:

The previous approach work just fine and passes the new test too. This is only to make things easier to reason about.

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • Documentation added
  • Tests updated
  • Title matches the required conventional commits format, see here
    • Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
  • Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
  • If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR
@Segflow
Segflow requested a review from a team as a code owner November 11, 2025 14:31
@Segflow

Segflow commented Nov 11, 2025

Copy link
Copy Markdown
Contributor Author

Benchmark shows it's slightly faster (but negligeable) :

Before the change (using mutex and cond)

goos: linux
goarch: amd64
pkg: github.com/grafana/loki/v3/pkg/ingester
cpu: AMD Ryzen 5 9600X 6-Core Processor             
BenchmarkReplayControllerBackpressure/without-backpressure-12         	 1134704	       978.1 ns/op	         0 total_flushes	       0 B/op	       0 allocs/op
BenchmarkReplayControllerBackpressure/with-backpressure-12            	    1066	   1108561 ns/op	         1.001 ops_per_flush	      1065 total_flushes	     905 B/op	      26 allocs/op
PASS
ok  	github.com/grafana/loki/v3/pkg/ingester	3.388s

After the change (using singleflight)

goos: linux
goarch: amd64
pkg: github.com/grafana/loki/v3/pkg/ingester
cpu: AMD Ryzen 5 9600X 6-Core Processor             
BenchmarkReplayControllerBackpressure/without-backpressure-12         	 1313337	       896.6 ns/op	         0 total_flushes	       0 B/op	       0 allocs/op
BenchmarkReplayControllerBackpressure/with-backpressure-12            	    1050	   1125361 ns/op	         1.001 ops_per_flush	      1049 total_flushes	     696 B/op	      15 allocs/op
PASS
ok  	github.com/grafana/loki/v3/pkg/ingester	3.219s
@Segflow Segflow changed the title refactor: Use singleflight to ensure only one inflight flush is happening Nov 11, 2025

@chaudum chaudum left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Segflow
Segflow merged commit 347f2f7 into main Nov 12, 2025
65 checks passed
@Segflow
Segflow deleted the refactor-ingester-replay-controll-use-singleflight branch November 12, 2025 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2 participants