feat(helm): Make access modes for persistence on all PVCs and StatefulSets editable

[petarnikolovski]

What this PR does / why we need it:

In case of self-hosted Kubernetes infrastructure there may be problems with RWO and StatefulSet. In our case, we need to have RWX so we can avoid multi-attach volume issues when node goes down. This PR introduces access mode in a backward compatible manner, leaving the option editable for chart users.

Which issue(s) this PR fixes:
N/A

Special notes for your reviewer:
N/A

Checklist

• Reviewed the CONTRIBUTING.md guide (required)
• Documentation added
• Tests updated
• Title matches the required conventional commits format, see here
  • Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
• Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
• For Helm chart changes bump the Helm chart version in production/helm/loki/Chart.yaml and update production/helm/loki/CHANGELOG.md and production/helm/loki/README.md. Example PR
• If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

[CLAassistant]

All committers have signed the CLA.

[hervenicol]

[disclaimer: I'm not a maintainer, just having a quick look out of curiosity]

The code looks good to me.

But you should re-generate the doc (make helm-docs IIRC) so it shows your new value. This should be caught by CI when it runs.

[petarnikolovski]

@hervenicol Thank you! Will do ☺️

[jkroepke]

Hi, thanks for your PR. could you add this on all PVCs and Statefulsets?

[petarnikolovski]

@jkroepke Hi, I have added access mode option to all StatefulSets.

[jkroepke]

Thanks for your PR, please also revert any non helm related changes, like in go.mod .

And use feat(helm) as PR title prefix

[petarnikolovski]

@jkroepke All issues are resolved. ☺️

[jkroepke]

LGTM

[JStickler]

@petarnikolovski could you take a look at the merge conflicts and get them resolved?

[Jayclifford345]

Hi @petarnikolovski, my apolgies would you mind merging with main. This should clear up the workflow errors and we can look to get this merged

[petarnikolovski]

@Jayclifford345 @JStickler I've updated the branch to most recent version.

[Jayclifford345]

Will annotate changelog with breaking change due to #19065