Skip to content

feat(helm): Update charts for GEL - #21590

Merged
JStickler merged 4 commits into
mainfrom
HelmGEL
Apr 23, 2026
Merged

feat(helm): Update charts for GEL#21590
JStickler merged 4 commits into
mainfrom
HelmGEL

Conversation

@JStickler

@JStickler JStickler commented Apr 16, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

Changes the installation default from Loki to GEL.
Updates the GEL version to the latest GEL release (3.6.6)
Updates the Minio repo to https://github.com/pgsty/minio/ as this fork has:

  1. Patched the critical CVEs that were left in the last Minio image.
  2. Restored the Admin console that Minio removed.
  3. Builds binary and docker images that Minio removed.
  4. Restored the community edition of the docs.

See this blog post for more information about my choice of replacement for Minio.

Which issue(s) this PR fixes:
Responding to a request on Slack from Support to update the GEL Helm chart.

Special notes for your reviewer:

This update was generated with AI and checked with a different model. I chose the most conservative options offered by the models, but this still needs human review by someone with more Helm experience than I have.


Note

Medium Risk
Changes Helm defaults for enterprise deployments and swaps the MinIO image source, which can affect production log storage behavior and upgrade outcomes if users rely on the previous provisioner default or MinIO images.

Overview
Updates the Loki Helm chart’s enterprise defaults by bumping Grafana Enterprise Logs to 3.6.6 (both enterprise.version and enterprise.image.tag) and clarifying enablement/license comments.

Switches the bundled minio subchart image overrides to the Pigsty fork (docker.io/pgsty/minio and docker.io/pgsty/mc) to mitigate an upstream MinIO CVE, and changes the enterprise provisioner to be disabled by default with new guidance on the required external admin token secret.

Documents these changes in CHANGELOG.md.

Reviewed by Cursor Bugbot for commit 3d2f670. Bugbot is set up for automated code reviews on this repo. Configure here.

@JStickler
JStickler requested a review from a team as a code owner April 16, 2026 22:39
@github-actions

This comment has been minimized.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Autofix Details

Bugbot Autofix prepared fixes for both issues found in the latest run.

  • ✅ Fixed: Chart appVersion mismatches OSS Loki image tag
    • Updated Chart appVersion to 3.6.7 and corrected NOTES to show the actual image/version and enterprise tokengen image.
  • ✅ Fixed: Default values now fail helm template validation
    • Changed default enterprise.provisioner.enabled to false so validation no longer fails with default values.

Create PR

Or push these changes by commenting:

@cursor push 8bfa510740
Preview (8bfa510740)
diff --git a/production/helm/loki/Chart.yaml b/production/helm/loki/Chart.yaml
--- a/production/helm/loki/Chart.yaml
+++ b/production/helm/loki/Chart.yaml
@@ -2,7 +2,7 @@
 name: loki
 description: Helm chart for Grafana Loki and Grafana Enterprise Logs supporting monolithic, simple scalable, and microservices modes.
 type: application
-appVersion: 3.6.6
+appVersion: 3.6.7
 version: 6.56.0
 home: https://grafana.github.io/helm-charts
 sources:

diff --git a/production/helm/loki/templates/NOTES.txt b/production/helm/loki/templates/NOTES.txt
--- a/production/helm/loki/templates/NOTES.txt
+++ b/production/helm/loki/templates/NOTES.txt
@@ -2,7 +2,7 @@
  Welcome to Grafana Loki
  Chart version: {{ .Chart.Version }}
  Chart Name: {{ .Chart.Name }}
- Loki version: {{ .Chart.AppVersion }}
+Loki version: {{- if .Values.enterprise.enabled -}}{{ .Values.enterprise.image.tag | default .Values.enterprise.version }}{{- else -}}{{ .Values.loki.image.tag | default .Chart.AppVersion }}{{- end }}
 ***********************************************************************
 
 ** Please be patient while the chart is being deployed **
@@ -166,7 +166,7 @@
 The provisioner job requires an admin token secret to be created manually.
 
 1. Generate an admin token using the Loki CLI:
-   docker run grafana/loki:{{ .Chart.AppVersion }} -target=tokengen -tokengen.token-file=/tmp/token
+   docker run {{ include "loki.enterpriseImage" . }} -target=tokengen -tokengen.token-file=/tmp/token
    
 2. Create the admin token secret:
    kubectl create secret generic {{ .Values.enterprise.adminToken.secret | default (printf "%s-admin-token" (include "loki.name" . )) }} \

diff --git a/production/helm/loki/values.yaml b/production/helm/loki/values.yaml
--- a/production/helm/loki/values.yaml
+++ b/production/helm/loki/values.yaml
@@ -691,7 +691,7 @@
   # Note: Uses enterprise.adminToken.secret value to mount the admin token used to call the admin api.
   provisioner:
     # -- Whether the job should be part of the deployment
-    enabled: true
+    enabled: false
     # -- Name of the secret to store provisioned tokens in
     provisionedSecretPrefix: null
     # -- Hook type(s) to customize when the job runs.  defaults to post-install

You can send follow-ups to the cloud agent here.

Comment thread production/helm/loki/Chart.yaml Outdated
Comment thread production/helm/loki/values.yaml Outdated
@github-actions

This comment has been minimized.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Chart appVersion not updated despite changelog claiming so
    • Updated production/helm/loki/Chart.yaml appVersion to 3.6.6 to align with CHANGELOG and README.

Create PR

Or push these changes by commenting:

@cursor push dde61afd4c
Preview (dde61afd4c)
diff --git a/production/helm/loki/Chart.yaml b/production/helm/loki/Chart.yaml
--- a/production/helm/loki/Chart.yaml
+++ b/production/helm/loki/Chart.yaml
@@ -2,7 +2,7 @@
 name: loki
 description: Helm chart for Grafana Loki and Grafana Enterprise Logs supporting monolithic, simple scalable, and microservices modes.
 type: application
-appVersion: 3.6.7
+appVersion: 3.6.6
 version: 6.56.0
 home: https://grafana.github.io/helm-charts
 sources:

You can send follow-ups to the cloud agent here.

Reviewed by Cursor Bugbot for commit cd08f4f. Configure here.

Comment thread production/helm/loki/Chart.yaml Outdated
@github-actions

This comment has been minimized.

Comment thread production/helm/loki/values.yaml
@QuentinBisson

Copy link
Copy Markdown
Contributor

I did not check about the specific fork because I do not really use/know minio.

Do you think it might be interesting to make this change a major version so the existing use base will need to take a look?

@JStickler

JStickler commented Apr 17, 2026

Copy link
Copy Markdown
Contributor Author

Do you think it might be interesting to make this change a major version

Good point, thanks!

@JStickler

JStickler commented Apr 23, 2026

Copy link
Copy Markdown
Contributor Author

Reverting back to Loki as the default, as integration tests are failing and I suspect they're configured to test Loki and not GEL.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Helm Diff Output - Summary

Default Values Scenario-diff-output
Distributed Scenario-diff-output
default, loki-release-minio, StatefulSet (apps) has changed:
  # Source: loki/charts/minio/templates/statefulset.yaml
  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: loki-release-minio
    labels:
      app: minio
      chart: minio-5.4.0
      release: loki-release
      heritage: Helm
  spec:
    updateStrategy:
      type: RollingUpdate
    podManagementPolicy: "Parallel"
    serviceName: loki-release-minio-svc
    replicas: 1
    selector:
      matchLabels:
        app: minio
        release: loki-release
    template:
      metadata:
        name: loki-release-minio
        labels:
          app: minio
          release: loki-release
        annotations:
          checksum/secrets: 83e00c080f13dd0c967e16625b3c5ce88fe138fb6e903ac49c54cd16f5f5b659
          checksum/config: 9e7d1b7d05afafdf1f4dd11b625c985b7bbaceea2255647f6c3c2dde5e257978
      spec:
        securityContext:
          fsGroup: 1000
          fsGroupChangePolicy: OnRootMismatch
          runAsGroup: 1000
          runAsUser: 1000
        serviceAccountName: minio-sa
        containers:
          - name: minio
-           image: quay.io/minio/minio:RELEASE.2024-12-18T13-15-44Z
+           image: docker.io/pgsty/minio:RELEASE.2026-03-14T12-00-00Z
            imagePullPolicy: IfNotPresent
            command: [
              "/bin/sh",
              "-ce",
              "/usr/bin/docker-entrypoint.sh minio server http://loki-release-minio-{0...0}.loki-release-minio-svc.default.svc/export-{0...1} -S /etc/minio/certs/ --address :9000 --console-address :9001"
            ]
            volumeMounts:
              - name: export-0
                mountPath: /export-0
              - name: export-1
                mountPath: /export-1            
            ports:
              - name: http
                containerPort: 9000
              - name: http-console
                containerPort: 9001
            env:
              - name: MINIO_ROOT_USER
                valueFrom:
                  secretKeyRef:
                    name: loki-release-minio
                    key: rootUser
              - name: MINIO_ROOT_PASSWORD
                valueFrom:
                  secretKeyRef:
                    name: loki-release-minio
                    key: rootPassword
              - name: MINIO_PROMETHEUS_AUTH_TYPE
                value: "public"
            resources:
              requests:
                cpu: 100m
                memory: 128Mi
            securityContext: 
              readOnlyRootFilesystem: false      
        volumes:
          - name: minio-user
            secret:
              secretName: loki-release-minio        
    volumeClaimTemplates:
      - apiVersion: v1
        kind: PersistentVolumeClaim
        metadata:
          name: export-0
        spec:
          accessModes: [ "ReadWriteOnce" ]
          resources:
            requests:
              storage: 5Gi
      - apiVersion: v1
        kind: PersistentVolumeClaim
        metadata:
          name: export-1
        spec:
          accessModes: [ "ReadWriteOnce" ]
          resources:
            requests:
              storage: 5Gi
default, loki-release-minio-post-job, Job (batch) has changed:
  # Source: loki/charts/minio/templates/post-job.yaml
  apiVersion: batch/v1
  kind: Job
  metadata:
    name: loki-release-minio-post-job
    labels:
      app: minio-post-job
      chart: minio-5.4.0
      release: loki-release
      heritage: Helm
    annotations:
      "helm.sh/hook": post-install,post-upgrade
      "helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation
  spec:
    template:
      metadata:
        labels:
          app: minio-job
          release: loki-release
      spec:
        restartPolicy: OnFailure      
        volumes:
          - name: etc-path
            emptyDir: {}
          - name: tmp
            emptyDir: {}
          - name: minio-configuration
            projected:
              sources:
                - configMap:
                    name: loki-release-minio
                - secret:
                    name: loki-release-minio
        serviceAccountName: minio-sa
        containers:
          - name: minio-make-bucket
-           image: "quay.io/minio/mc:RELEASE.2024-11-21T17-21-54Z"
+           image: "docker.io/pgsty/mc:RELEASE.2026-03-13T08-57-32Z"
            imagePullPolicy: IfNotPresent
            command: [ "/bin/sh", "/config/initialize" ]
            env:
              - name: MINIO_ENDPOINT
                value: loki-release-minio
              - name: MINIO_PORT
                value: "9000"
            volumeMounts:
              - name: etc-path
                mountPath: /etc/minio/mc
              - name: tmp
                mountPath: /tmp
              - name: minio-configuration
                mountPath: /config
            resources:
              requests:
                memory: 128Mi
          - name: minio-make-user
-           image: "quay.io/minio/mc:RELEASE.2024-11-21T17-21-54Z"
+           image: "docker.io/pgsty/mc:RELEASE.2026-03-13T08-57-32Z"
            imagePullPolicy: IfNotPresent
            command: [ "/bin/sh", "/config/add-user" ]
            env:
              - name: MINIO_ENDPOINT
                value: loki-release-minio
              - name: MINIO_PORT
                value: "9000"
            volumeMounts:
              - name: etc-path
                mountPath: /etc/minio/mc
              - name: tmp
                mountPath: /tmp
              - name: minio-configuration
                mountPath: /config
            resources:
              requests:
                memory: 128Mi
Ingress Values Scenario-diff-output
Legacy Monitoring Values Scenario-diff-output
Simple Scalable AWS Kube IRSA Values Scenario-diff-output
760c760
<           image: "docker.io/grafana/enterprise-logs:3.6.6"
---
>           image: "docker.io/grafana/enterprise-logs:3.6.7"
865c865
<           image: docker.io/grafana/enterprise-logs:3.6.6
---
>           image: docker.io/grafana/enterprise-logs:3.6.7
992c992
<           image: docker.io/grafana/enterprise-logs:3.6.6
---
>           image: docker.io/grafana/enterprise-logs:3.6.7
1417c1417
<           image: docker.io/grafana/enterprise-logs:3.6.6
---
>           image: docker.io/grafana/enterprise-logs:3.6.7
1492a1493,1552
> # Source: loki/templates/provisioner/serviceaccount-provisioner.yaml
> apiVersion: v1
> kind: ServiceAccount
> metadata:
>   name: enterprise-logs-provisioner
>   namespace: default
>   labels:
>     helm.sh/chart: loki-6.55.0
>     app.kubernetes.io/name: enterprise-logs
>     app.kubernetes.io/instance: loki-release
>     app.kubernetes.io/version: "3.6.7"
>     app.kubernetes.io/component: provisioner
>   annotations:
>     eks.amazonaws.com/role-arn: arn:aws:iam::2222222:role/test-role
>     "helm.sh/hook": "post-install"
> ---
> # Source: loki/templates/provisioner/role-provisioner.yaml
> apiVersion: rbac.authorization.k8s.io/v1
> kind: ClusterRole
> metadata:
>   name: enterprise-logs-provisioner
>   namespace: default
>   labels:
>     helm.sh/chart: loki-6.55.0
>     app.kubernetes.io/name: enterprise-logs
>     app.kubernetes.io/instance: loki-release
>     app.kubernetes.io/version: "3.6.7"
>     app.kubernetes.io/component: provisioner
>   annotations:
>     eks.amazonaws.com/role-arn: arn:aws:iam::2222222:role/test-role
>     "helm.sh/hook": "post-install"
> rules:
>   - apiGroups: [""]
>     resources: ["secrets"]
>     verbs: ["create"]
> ---
> # Source: loki/templates/provisioner/rolebinding-provisioner.yaml
> apiVersion: rbac.authorization.k8s.io/v1
> kind: ClusterRoleBinding
> metadata:
>   name: enterprise-logs-provisioner
>   namespace: default
>   labels:
>     helm.sh/chart: loki-6.55.0
>     app.kubernetes.io/name: enterprise-logs
>     app.kubernetes.io/instance: loki-release
>     app.kubernetes.io/version: "3.6.7"
>     app.kubernetes.io/component: provisioner
>   annotations:
>     eks.amazonaws.com/role-arn: arn:aws:iam::2222222:role/test-role
>     "helm.sh/hook": "post-install"
> roleRef:
>   apiGroup: rbac.authorization.k8s.io
>   kind: ClusterRole
>   name: enterprise-logs-provisioner
> subjects:
>   - kind: ServiceAccount
>     name: enterprise-logs-provisioner
>     namespace: default
> ---
1520a1581,1651
> ---
> # Source: loki/templates/provisioner/job-provisioner.yaml
> apiVersion: batch/v1
> kind: Job
> metadata:
>   name: enterprise-logs-provisioner
>   namespace: default
>   labels:
>     helm.sh/chart: loki-6.55.0
>     app.kubernetes.io/name: enterprise-logs
>     app.kubernetes.io/instance: loki-release
>     app.kubernetes.io/version: "3.6.7"
>     app.kubernetes.io/component: provisioner
>   annotations:
>     eks.amazonaws.com/role-arn: arn:aws:iam::2222222:role/test-role
>     "helm.sh/hook": "post-install"
>     "helm.sh/hook-weight": "15"
> spec:
>   backoffLimit: 6
>   completions: 1
>   parallelism: 1
>   template:
>     metadata:
>       labels:
>         app.kubernetes.io/name: enterprise-logs
>         app.kubernetes.io/instance: loki-release
>         app.kubernetes.io/component: provisioner
>       annotations:
>         eks.amazonaws.com/role-arn: arn:aws:iam::2222222:role/test-role
>     spec:
>       securityContext:
>         fsGroup: 10001
>         runAsGroup: 10001
>         runAsNonRoot: true
>         runAsUser: 10001
>       containers:
>         - name: provisioner
>           image: us-docker.pkg.dev/grafanalabs-global/docker-enterprise-provisioner-prod/enterprise-provisioner:latest
>           imagePullPolicy: IfNotPresent
>           command:
>             - /bin/sh
>             - -exuc
>             - |
>               echo "================================================================================"
>               echo "Starting provisioner. Any configured tenants will be provisioned below."
>               echo "If you need to manually provision a tenant, see the following documentation: "
>               echo "https://github.com/grafana/loki/blob/main/production/helm/loki/docs/examples/enterprise/README.md"
>               echo "================================================================================"
>               echo "\nProvisioning self-monitoring tenant: self-monitoring..."
>               /usr/bin/provisioner \
>                 -cluster-name=loki-release \
>                 -api-url=http://loki-release-enterprise-logs-gateway.default.svc.cluster.local.:80 \
>                 -tenant=self-monitoring \
>                 -access-policy=self-monitoring:self-monitoring:logs:write,logs:read \
>                 -token=self-monitoring
>               echo "\n================================================================================"
>               echo "Provisioning complete."
>               echo "================================================================================"
>           volumeMounts:
>             - name: admin-token
>               mountPath: /bootstrap/token
>               subPath: token
>           securityContext:
>             null
>       restartPolicy: OnFailure
>       serviceAccount: enterprise-logs-provisioner
>       serviceAccountName: enterprise-logs-provisioner
>       volumes:
>         - name: admin-token
>           secret:
>             secretName: "loki-admin-token"
Simple Thanos Values Scenario-diff-output
1216c1216
<           image: "docker.io/grafana/enterprise-logs:3.6.6"
---
>           image: "docker.io/grafana/enterprise-logs:3.6.7"
1327c1327
<           image: docker.io/grafana/enterprise-logs:3.6.6
---
>           image: docker.io/grafana/enterprise-logs:3.6.7
1437c1437
<           image: docker.io/pgsty/minio:RELEASE.2026-03-14T12-00-00Z
---
>           image: quay.io/minio/minio:RELEASE.2024-12-18T13-15-44Z
1552c1552
<           image: docker.io/grafana/enterprise-logs:3.6.6
---
>           image: docker.io/grafana/enterprise-logs:3.6.7
1976c1976
<           image: docker.io/grafana/enterprise-logs:3.6.6
---
>           image: docker.io/grafana/enterprise-logs:3.6.7
2050a2051,2107
> # Source: loki/templates/provisioner/serviceaccount-provisioner.yaml
> apiVersion: v1
> kind: ServiceAccount
> metadata:
>   name: enterprise-logs-provisioner
>   namespace: default
>   labels:
>     helm.sh/chart: loki-6.55.0
>     app.kubernetes.io/name: enterprise-logs
>     app.kubernetes.io/instance: loki-release
>     app.kubernetes.io/version: "3.6.7"
>     app.kubernetes.io/component: provisioner
>   annotations:
>     "helm.sh/hook": "post-install"
> ---
> # Source: loki/templates/provisioner/role-provisioner.yaml
> apiVersion: rbac.authorization.k8s.io/v1
> kind: ClusterRole
> metadata:
>   name: enterprise-logs-provisioner
>   namespace: default
>   labels:
>     helm.sh/chart: loki-6.55.0
>     app.kubernetes.io/name: enterprise-logs
>     app.kubernetes.io/instance: loki-release
>     app.kubernetes.io/version: "3.6.7"
>     app.kubernetes.io/component: provisioner
>   annotations:
>     "helm.sh/hook": "post-install"
> rules:
>   - apiGroups: [""]
>     resources: ["secrets"]
>     verbs: ["create"]
> ---
> # Source: loki/templates/provisioner/rolebinding-provisioner.yaml
> apiVersion: rbac.authorization.k8s.io/v1
> kind: ClusterRoleBinding
> metadata:
>   name: enterprise-logs-provisioner
>   namespace: default
>   labels:
>     helm.sh/chart: loki-6.55.0
>     app.kubernetes.io/name: enterprise-logs
>     app.kubernetes.io/instance: loki-release
>     app.kubernetes.io/version: "3.6.7"
>     app.kubernetes.io/component: provisioner
>   annotations:
>     "helm.sh/hook": "post-install"
> roleRef:
>   apiGroup: rbac.authorization.k8s.io
>   kind: ClusterRole
>   name: enterprise-logs-provisioner
> subjects:
>   - kind: ServiceAccount
>     name: enterprise-logs-provisioner
>     namespace: default
> ---
2116c2173
<           image: "docker.io/pgsty/mc:RELEASE.2026-03-13T08-57-32Z"
---
>           image: "quay.io/minio/mc:RELEASE.2024-11-21T17-21-54Z"
2135c2192
<           image: "docker.io/pgsty/mc:RELEASE.2026-03-13T08-57-32Z"
---
>           image: "quay.io/minio/mc:RELEASE.2024-11-21T17-21-54Z"
2152a2210,2277
> ---
> # Source: loki/templates/provisioner/job-provisioner.yaml
> apiVersion: batch/v1
> kind: Job
> metadata:
>   name: enterprise-logs-provisioner
>   namespace: default
>   labels:
>     helm.sh/chart: loki-6.55.0
>     app.kubernetes.io/name: enterprise-logs
>     app.kubernetes.io/instance: loki-release
>     app.kubernetes.io/version: "3.6.7"
>     app.kubernetes.io/component: provisioner
>   annotations:
>     "helm.sh/hook": "post-install"
>     "helm.sh/hook-weight": "15"
> spec:
>   backoffLimit: 6
>   completions: 1
>   parallelism: 1
>   template:
>     metadata:
>       labels:
>         app.kubernetes.io/name: enterprise-logs
>         app.kubernetes.io/instance: loki-release
>         app.kubernetes.io/component: provisioner
>     spec:
>       securityContext:
>         fsGroup: 10001
>         runAsGroup: 10001
>         runAsNonRoot: true
>         runAsUser: 10001
>       containers:
>         - name: provisioner
>           image: us-docker.pkg.dev/grafanalabs-global/docker-enterprise-provisioner-prod/enterprise-provisioner:latest
>           imagePullPolicy: IfNotPresent
>           command:
>             - /bin/sh
>             - -exuc
>             - |
>               echo "================================================================================"
>               echo "Starting provisioner. Any configured tenants will be provisioned below."
>               echo "If you need to manually provision a tenant, see the following documentation: "
>               echo "https://github.com/grafana/loki/blob/main/production/helm/loki/docs/examples/enterprise/README.md"
>               echo "================================================================================"
>               echo "\nProvisioning self-monitoring tenant: self-monitoring..."
>               /usr/bin/provisioner \
>                 -cluster-name=loki-release \
>                 -api-url=http://loki-release-enterprise-logs-gateway.default.svc.cluster.local.:80 \
>                 -tenant=self-monitoring \
>                 -access-policy=self-monitoring:self-monitoring:logs:write,logs:read \
>                 -token=self-monitoring
>               echo "\n================================================================================"
>               echo "Provisioning complete."
>               echo "================================================================================"
>           volumeMounts:
>             - name: admin-token
>               mountPath: /bootstrap/token
>               subPath: token
>           securityContext:
>             null
>       restartPolicy: OnFailure
>       serviceAccount: enterprise-logs-provisioner
>       serviceAccountName: enterprise-logs-provisioner
>       volumes:
>         - name: admin-token
>           secret:
>             secretName: "loki-admin-token"
Single Binary Scenario-diff-output
@JStickler
JStickler merged commit 733fb71 into main Apr 23, 2026
213 of 217 checks passed
@JStickler
JStickler deleted the HelmGEL branch April 23, 2026 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3 participants