[blocklist]: replace O(N²) slice scans in updateInternal with map lookups - #7140
Merged
zalegrala merged 3 commits intoJun 2, 2026
Merged
Conversation
zalegrala
force-pushed
the
zl/blocklist-updateinternal-o-n2-fix
branch
2 times, most recently
from
May 14, 2026 20:06
d076b9e to
7648e1a
Compare
zalegrala
marked this pull request as ready for review
May 14, 2026 21:45
zalegrala
requested review from
carles-grafana,
electron0zero,
ie-pham,
javiermolinar,
mapno,
mattdurham,
mdisibio,
oleg-kozlyuk-grafana,
ruslan-mikhailov,
stoewer and
yvrhdn
as code owners
May 14, 2026 21:45
Contributor
There was a problem hiding this comment.
Pull request overview
Replaces four slices.ContainsFunc linear scans in List.updateInternal with pre-built map[backend.UUID]struct{} lookups, turning the regular-blocks add phase from O(N·M) to O(N+M). The benchmark (BenchmarkUpdateInternalLargeAdd) demonstrates ~53× speedup on a 100K-existing + 1K-add workload. Logic for the compacted-blocks branch is similarly converted, preserving the original semantics (compacted-add still dedupes against the pre-existing set only, matching prior behavior).
Changes:
- Build
removeIDs,compactedAddIDs,compactedRemoveIDs, andfinalIDsmaps once and use map membership checks in both the regular and compacted update loops. - Drop the
slicesimport and thehasID/hasIDCclosure helpers. - Add
BenchmarkUpdateInternalLargeAddto guard against future regressions.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| tempodb/blocklist/list.go | Replace O(N²) slice scans in updateInternal with map-based lookups; logic remains equivalent to the prior implementation. |
| tempodb/blocklist/list_test.go | New benchmark exercising the large-add path (100K existing + 1K adds) to demonstrate and protect the perf improvement. |
zalegrala
force-pushed
the
zl/blocklist-updateinternal-o-n2-fix
branch
from
May 14, 2026 21:57
7648e1a to
46346d4
Compare
Replace slices.ContainsFunc calls with map-based O(1) lookups throughout updateInternal. The add loop previously scanned the full `final` slice on every iteration — 1K new blocks × 100K existing = 100M comparisons per tenant per poll cycle. All four slice scans (remove, compactedAdd, compactedRemove, and final-dedup) are now O(1) map lookups built once before the loops. BenchmarkUpdateInternalLargeAdd (100K existing, 1K adds): before: 236,473,481 ns/op after: 4,433,651 ns/op (~53× faster)
zalegrala
force-pushed
the
zl/blocklist-updateinternal-o-n2-fix
branch
from
June 2, 2026 18:24
9ab0f77 to
40cf814
Compare
electron0zero
approved these changes
Jun 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does:
Optimizes
updateInternal(tempodb/blocklist/list.go). The add loop calledslices.ContainsFunc(final, ...)on every incoming block — scanning up to 100K existing entries per block added. With ~1K new blocks per compaction cycle, that's 100M UUID comparisons per tenant per poll.All four
slices.ContainsFuncscans (remove,compactedAdd,compactedRemove, and thefinaldedup check) are replaced withmap[backend.UUID]struct{}lookups built once before the loops, reducing the add phase from O(N·M) to O(N+M).Benchmark (
BenchmarkUpdateInternalLargeAdd— 100K existing blocks, 1K new adds):~53× faster per call. The alloc count increase reflects map bucket overhead for the 100K-capacity
finalIDsmap; memory stays under 4MB per call.Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]