Skip to content

feat(ingester): implement partition shuffle sharding for ingester - #14304

Merged
cyriltovena merged 1 commit into
grafana:mainfrom
cyriltovena:ingester-partition-shuffle-sharding
Oct 2, 2024
Merged

feat(ingester): implement partition shuffle sharding for ingester#14304
cyriltovena merged 1 commit into
grafana:mainfrom
cyriltovena:ingester-partition-shuffle-sharding

Conversation

@cyriltovena

@cyriltovena cyriltovena commented Sep 29, 2024

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

This add shuffle sharding per tenant to limit partition they will use in kafka mode.

We might need to consider a ring change when a tenant shard size change, not implemented yet.

Which issue(s) this PR fixes:
Fixes https://github.com/grafana/loki-private/issues/1131

Special notes for your reviewer:

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • Documentation added
  • Tests updated
  • Title matches the required conventional commits format, see here
    • Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
  • Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
  • For Helm chart changes bump the Helm chart version in production/helm/loki/Chart.yaml and update production/helm/loki/CHANGELOG.md and production/helm/loki/README.md. Example PR
  • If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR
@cyriltovena
cyriltovena requested a review from a team as a code owner September 29, 2024 12:19
@github-actions github-actions Bot added the type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories label Sep 29, 2024

@benclive benclive left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice :) I'll borrow this in the Limiter's partition implementation.

@cyriltovena
cyriltovena merged commit 1a4436c into grafana:main Oct 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories

2 participants