Skip to content

perf(tsdb): don't fetch labels in TSDBIndex.GetChunkRefs - #20970

Merged
paul1r merged 4 commits into
grafana:mainfrom
bboreham:chunkrefs-skiplabels
Feb 25, 2026
Merged

perf(tsdb): don't fetch labels in TSDBIndex.GetChunkRefs#20970
paul1r merged 4 commits into
grafana:mainfrom
bboreham:chunkrefs-skiplabels

Conversation

@bboreham

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

I noticed in a profile from production that index-gateways were spending a lot of time reading labels, in a context where labels are not typically needed.

image

This PR is made up of a refactoring to simplify the change, then adding the possibility to not fetch labels, and finally changing GetChunkRefs to only fetch labels when it needs to. It may need labels for series filtering, which is used by some downstream projects.

The benchmark results are not impressive, because BenchmarkTSDBIndex_GetChunkRefs only creates three series.
Still, a 30% drop in objects allocated shows that something was achieved.

I fixed the benchmark to return memory to the pool, otherwise that dominates.

goos: linux
goarch: amd64
pkg: github.com/grafana/loki/v3/pkg/storage/stores/shipper/indexshipper/tsdb
cpu: Intel(R) Core(TM) i7-14700K
                          │ before.txt  │             after.txt             │
                          │   sec/op    │   sec/op     vs base              │
TenantHeads/10-28           105.1µ ± 3%   106.4µ ± 2%       ~ (p=0.093 n=6)
TenantHeads/100-28          775.7µ ± 4%   717.7µ ± 5%  -7.49% (p=0.002 n=6)
TenantHeads/1000-28         7.897m ± 6%   7.650m ± 3%       ~ (p=0.132 n=6)
IndexClient_Stats-28        17.55µ ± 1%   17.67µ ± 1%       ~ (p=0.093 n=6)
TSDBIndex_GetChunkRefs-28   860.3µ ± 1%   836.5µ ± 2%  -2.77% (p=0.009 n=6)
TSDBIndex_Volume-28         280.0µ ± 1%   280.2µ ± 1%       ~ (p=0.937 n=6)
geomean                     373.7µ        366.4µ       -1.95%

                          │  before.txt  │             after.txt              │
                          │     B/op     │     B/op      vs base              │
TenantHeads/10-28           627.4Ki ± 0%   627.1Ki ± 0%       ~ (p=0.093 n=6)
TenantHeads/100-28          6.094Mi ± 0%   6.077Mi ± 0%  -0.28% (p=0.002 n=6)
TenantHeads/1000-28         63.20Mi ± 1%   63.33Mi ± 0%       ~ (p=0.818 n=6)
IndexClient_Stats-28        2.685Ki ± 2%   2.700Ki ± 2%       ~ (p=0.485 n=6)
TSDBIndex_GetChunkRefs-28   1.343Mi ± 0%   1.342Mi ± 0%  -0.07% (p=0.002 n=6)
TSDBIndex_Volume-28         67.22Ki ± 0%   67.22Ki ± 0%       ~ (p=0.061 n=6)
geomean                     630.6Ki        631.0Ki       +0.07%

                          │ before.txt  │              after.txt               │
                          │  allocs/op  │  allocs/op   vs base                 │
TenantHeads/10-28           2.135k ± 0%   2.125k ± 0%   -0.47% (p=0.002 n=6)
TenantHeads/100-28          21.34k ± 0%   21.23k ± 0%   -0.49% (p=0.002 n=6)
TenantHeads/1000-28         213.5k ± 0%   212.5k ± 0%   -0.47% (p=0.002 n=6)
IndexClient_Stats-28         55.00 ± 0%    55.00 ± 0%        ~ (p=1.000 n=6) ¹
TSDBIndex_GetChunkRefs-28    27.00 ± 0%    19.00 ± 0%  -29.63% (p=0.002 n=6)
TSDBIndex_Volume-28         3.060k ± 0%   3.060k ± 0%        ~ (p=1.000 n=6) ¹
geomean                     1.880k        1.769k        -5.91%
¹ all samples are equal

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • NA Documentation added
  • NA Tests updated
  • Title matches the required conventional commits format, see here
  • NA Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
  • NA If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory.
This lets the function under test perform best.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Also stop passing chks parameter which is unused.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Unless necessary for filtering.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
@bboreham
bboreham requested a review from a team as a code owner February 25, 2026 19:57
@paul1r
paul1r merged commit d94ee60 into grafana:main Feb 25, 2026
87 checks passed
bboreham added a commit to bboreham/loki that referenced this pull request Feb 26, 2026
This is the same change as grafana#20969,
missed from grafana#20970 by accident.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
trevorwhitney pushed a commit that referenced this pull request May 15, 2026
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
sophiewaldman pushed a commit that referenced this pull request May 15, 2026
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
@bboreham
bboreham deleted the chunkrefs-skiplabels branch June 19, 2026 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2 participants