helm: Added trafficDistribution configuration option to distributor, gateway, querier, query-frontend, query-scheduler, ingester, and store-gateway services

[Mohijeet]

What this PR does

This Pull Request (PR) introduces a significant enhancement for specific use cases where a metrics forwarder and Mimir instances are communicating within the same Kubernetes cluster. Currently, there's no native configuration allowing Mimir distributors and ingesters to prioritise same-Availability Zone (AZ) communication for data ingestion, even when cross-AZ replication is disabled.

This PR aims to enable zone-aware data transfer to optimise data ingestion by leveraging existing Kubernetes kube-proxy features for traffic distribution (as described here). By allowing the metrics agent to communicate with same-AZ endpoints of the distributor, NGINX, or gateway (based on configuration), we can achieve more efficient and potentially lower-latency data transfer. This is particularly beneficial for reducing cross-AZ traffic costs and improving overall performance within the cluster

(I added this in readpath to keep things consistent in the querier or querier_frontend during data reads. In the end, the data will still be fetched from Ingesters across different availability zones.)

Which issue(s) this PR fixes or relates to

#10214

Fixes #10214

Checklist

• Tests updated.
• Documentation added.
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]. If changelog entry is not needed, please add the changelog-not-needed label to the PR.
• about-versioning.md updated with experimental features.

[CLAassistant]

All committers have signed the CLA.

[frittentheke]

Thanks for this PR, I was just about to even open an issue for this to be added to the charts.
You might want to fix the link (#fixes) to have your PR actually how up as fix for #10214

[Mohijeet]

Hi, I’ve updated the #fixes link based on my understanding.
Since this is my first PR, could you let me know the best way to request a review and proceed?
I really want to use this feature with helm since i need to make changes manually every time for rollout or update...

[Shreyank031]

This is a great feature.
Looking forward for the pr to be merged.

[dimitarvdimitrov]

can you also add an entry to the CHANGELOG.md file in operations/helm/charts/mimir-distributed?

[dimitarvdimitrov]

thank you for the contribution @Mohijeet can you please sign the CLA? I think it's possible that some of the commits you've pushed are with a different author than the one in your github profile (which supposedly signed the CLA). Can you check again? This is the only thing blocking merging this now.

If you need to re-push commits that's also ok, just be careful to also include the diff that I pushed a few minutes ago.