Kubernetes.audit_logs: fix processing of Azure AKS audit logs

[chemamartinez]

Proposed commit message

Azure AKS audit logs forwarded through Azure Eventhub are wrapped with an Azure envelope that follows this schema.

This PR fixes the audit_logs ingest pipeline to extract the audit log from the Azure resource log, as well as adding the Azure metadata as a new object called aks_metadata.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Related issues

• Added Azure AKS support at Kubernetes.audit_logs: add support for cloud providers #14554

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[zmoog]

Looks good at first glance. I'm trying it out on my local cluster.

[zmoog]

@chemamartinez, I set up a Diagnostic Setting to send "Kubernetes Audit" and "Kubernetes Audit Admin Logs" to an event hub:

And I deployed an Elastic Agent to pull these logs.

Elasticsearch is rejecting all the logs due to this error:

{
    "log.level": "warn",
    "@timestamp": "2025-10-07T16:45:33.946Z",
    "message": "Cannot index event '{\"@timestamp\":\"2025-10-07T16:45:33.929Z\",\"tags\":[\"forwarded\",\"kubernetes-audit_logs\"],\"data_stream\":{\"dataset\":\"kubernetes.audit_logs\",\"namespace\":\"default\",\"type\":\"logs\"},\"ecs\":{\"version\":\"8.0.0\"},\"input\":{\"type\":\"azure-eventhub\"},\"event\":{\"dataset\":\"kubernetes.audit_logs\"},\"elastic_agent\":{\"version\":\"8.18.8\",\"id\":\"15ec9ad7-5eee-48d4-a690-5975b00d008b\",\"snapshot\":false},\"agent\":{\"id\":\"15ec9ad7-5eee-48d4-a690-5975b00d008b\",\"version\":\"8.18.8\",\"ephemeral_id\":\"505ce226-9b72-453b-b908-3b981413de58\",\"name\":\"docker-fleet-agent\",\"type\":\"filebeat\"},\"message\":\"{\\\"category\\\":\\\"kube-audit-admin\\\",\\\"operationName\\\":\\\"Microsoft.ContainerService/managedClusters/diagnosticLogs/Read\\\",\\\"properties\\\":{\\\"containerID\\\":\\\"a64cba7fefbf5020788dc29d9247157585bbb64826bf7209623ca7bb49b15fe7\\\",\\\"log\\\":\\\"{\\\\\\\"kind\\\\\\\":\\\\\\\"Event\\\\\\\",\\\\\\\"apiVersion\\\\\\\":\\\\\\\"audit.k8s.io/v1\\\\\\\",\\\\\\\"level\\\\\\\":\\\\\\\"Metadata\\\\\\\",\\\\\\\"auditID\\\\\\\":\\\\\\\"bfbd9d4d-ce5c-4ead-a743-af5b12b287f2\\\\\\\",\\\\\\\"stage\\\\\\\":\\\\\\\"ResponseComplete\\\\\\\",\\\\\\\"requestURI\\\\\\\":\\\\\\\"/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kubelet-serving-csr-approver\\\\\\\",\\\\\\\"verb\\\\\\\":\\\\\\\"update\\\\\\\",\\\\\\\"user\\\\\\\":{\\\\\\\"username\\\\\\\":\\\\\\\"aksService\\\\\\\",\\\\\\\"groups\\\\\\\":[\\\\\\\"system:masters\\\\\\\",\\\\\\\"system:authenticated\\\\\\\"]},\\\\\\\"sourceIPs\\\\\\\":[\\\\\\\"172.31.51.172\\\\\\\"],\\\\\\\"userAgent\\\\\\\":\\\\\\\"approver/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election\\\\\\\",\\\\\\\"objectRef\\\\\\\":{\\\\\\\"resource\\\\\\\":\\\\\\\"leases\\\\\\\",\\\\\\\"namespace\\\\\\\":\\\\\\\"kube-system\\\\\\\",\\\\\\\"name\\\\\\\":\\\\\\\"kubelet-serving-csr-approver\\\\\\\",\\\\\\\"uid\\\\\\\":\\\\\\\"14be99f8-ebb7-47b9-a194-43e63d9386af\\\\\\\",\\\\\\\"apiGroup\\\\\\\":\\\\\\\"coordination.k8s.io\\\\\\\",\\\\\\\"apiVersion\\\\\\\":\\\\\\\"v1\\\\\\\",\\\\\\\"resourceVersion\\\\\\\":\\\\\\\"93066366\\\\\\\"},\\\\\\\"responseStatus\\\\\\\":{\\\\\\\"metadata\\\\\\\":{},\\\\\\\"code\\\\\\\":200},\\\\\\\"requestReceivedTimestamp\\\\\\\":\\\\\\\"2025-10-07T15:44:35.237083Z\\\\\\\",\\\\\\\"stageTimestamp\\\\\\\":\\\\\\\"2025-10-07T15:44:35.241488Z\\\\\\\",\\\\\\\"annotations\\\\\\\":{\\\\\\\"authorization.k8s.io/decision\\\\\\\":\\\\\\\"allow\\\\\\\",\\\\\\\"authorization.k8s.io/reason\\\\\\\":\\\\\\\"\\\\\\\"}}\\\",\\\"pod\\\":\\\"kube-apiserver-869d7bb754-kkg69\\\",\\\"stream\\\":\\\"stdout\\\"},\\\"resourceId\\\":\\\"/SUBSCRIPTIONS/[redacted]/RESOURCEGROUPS/MKATSOULIS/PROVIDERS/MICROSOFT.CONTAINERSERVICE/MANAGEDCLUSTERS/MKATSOULIS-CLUSTER\\\",\\\"serviceBuild\\\":\\\"na\\\",\\\"time\\\":\\\"2025-10-07T15:44:35.241642842Z\\\"}\",\"azure\":{\"eventhub\":\"logs\",\"consumer_group\":\"$Default\",\"offset\":31619555306984,\"sequence_number\":69292040,\"enqueued_time\":\"2025-10-07T16:45:33.190Z\"}}\n, Meta: {\"input_id\":\"azure-eventhub-audit-logs-632b5fc9-7f73-44d2-83e7-93f889db311e\",\"raw_index\":\"logs-kubernetes.audit_logs-default\",\"stream_id\":\"azure-eventhub-kubernetes.audit_logs-632b5fc9-7f73-44d2-83e7-93f889db311e\"}' (status=400): {\"type\":\"document_parsing_exception\",\"reason\":\"[1:24] object mapping for [kubernetes.audit] tried to parse field [audit] as object, but found a concrete value\"}, dropping event!",
    "component": {
        "binary": "filebeat",
        "dataset": "elastic_agent.filebeat",
        "id": "azure-eventhub-default",
        "type": "azure-eventhub"
    },
    "log": {
        "source": "azure-eventhub-default"
    },
    "log.logger": "elasticsearch",
    "log.origin": {
        "file.line": 528,
        "file.name": "elasticsearch/client.go",
        "function": "github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.(*Client).applyItemStatus"
    },
    "service.name": "filebeat",
    "log.type": "event",
    "ecs.version": "1.6.0",
    "ecs.version": "1.6.0"
}

[chemamartinez]

@zmoog probably for some format reason the properties.log is being treated as a string when the field is processed by the pipeline.

I managed to solve it by decoding it as a JSON again. Now logs are being ingested as expected.

Another weird problem is that I could not reproduce it with pipeline tests, no matter how the input log looks like. I finally decided to keep the original logic (rename and json processors for the same field). It may not be the most elegant solution, but at least we can be sure it will work in all cases, so I think it's the safest choice.

On the other hand, I added mappings for all unknown metadata fields.

[zmoog]

@chemamartinez, I hope you don't mind me adding a test case for the kube-audit log category that includes a properties.log field as a string with embedded JSON.

[zmoog]

LGTM

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: be391e4

History

• 💔 Build #32512 failed b2c1e14
• 💚 Build #32416 succeeded 767a39b

cc @chemamartinez

[elastic-vault-github-plugin-prod]

Package kubernetes - 1.81.1 containing this change is available at https://epr.elastic.co/package/kubernetes/1.81.1/