Skip to content

Add support for ingester query shards - #3574

Merged
zalegrala merged 21 commits into
grafana:mainfrom
zalegrala:ingesterQueryShards
Apr 18, 2024
Merged

Add support for ingester query shards#3574
zalegrala merged 21 commits into
grafana:mainfrom
zalegrala:ingesterQueryShards

Conversation

@zalegrala

@zalegrala zalegrala commented Apr 15, 2024

Copy link
Copy Markdown
Contributor

What this PR does:

This change allows the search queries against the ingesters to be broken into smaller queries based on the time span.

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]
Comment thread modules/frontend/search_sharder.go
Comment thread modules/frontend/search_sharder.go Outdated
Comment thread modules/frontend/config.go
@joe-elliott joe-elliott mentioned this pull request Apr 17, 2024
9 tasks
@zalegrala
zalegrala force-pushed the ingesterQueryShards branch from 0793eee to 42a9612 Compare April 17, 2024 15:48
Comment thread modules/frontend/config.go
Comment thread modules/frontend/search_sharder.go
Comment thread modules/frontend/search_sharder.go
Comment thread modules/frontend/search_sharder_test.go Outdated
Comment thread .golangci.yml
Comment thread modules/frontend/search_sharder.go

@knylander-grafana knylander-grafana left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for adding updates to the doc.

@zalegrala
zalegrala merged commit 5e26604 into grafana:main Apr 18, 2024
@zalegrala
zalegrala deleted the ingesterQueryShards branch April 18, 2024 17:40
mattdurham pushed a commit to mattdurham/tempo that referenced this pull request Jun 18, 2026
* Add support for ingester query shards

* Update changelog

* Drop unused test method

* Avoid boring lint error

* Fix /0 panic

* Fix first query param in test

* Drop TODO after fixed in config

* Tidy

* Test for case where we skip doing too much work

* Reduce shards until we are above the hardcode 1m interval threshold

* Test greater than query count

* Tidy

* Avoid /0

* Require 5 occurrences for goconst

* Set goconst to ignore tests, drop previous config

* Avoid shard count checking and set interval floor

* Clean and lint

* Handle case where more shards than the window will tollerate

* Ensure tests handle delta in both postivie and negative directions

* Grow last shard to fill the gap and add test

* Tidy
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

4 participants