Describe the bug
Spans that use empty string as key for resource attribute break remote write for span metrics.
Example:
{
"value": "foo",
"key": ""
}
To Reproduce
Steps to reproduce the behavior:
- Write span with attribute key as empty string
- Enable span metrics generator with traces_target_info enabled.
Expected behavior
Ideally, the metrics generator would ignore invalid label names which will be rejected by prometheus.
Environment:
Additional Context
ts=2025-05-21T17:44:50.854871485Z caller=mapping_proxy.go:323 level=error instance_type=traces user=*** metrics=*** target=http://*.svc.cluster.local/api/prom/push msg="status code not 2xx while proxying" code="400 Bad Request" response="received a series with an invalid label: '' series: 'traces_target_info{=\"foo\", __metrics_gen_instance=\"metrics-generator-3\", application=\"app\", cluster=\"cluster\", container_id=\"123456' (err-mimir-label-invalid)\n"
Describe the bug
Spans that use empty string as key for resource attribute break remote write for span metrics.
Example:
{
"value": "foo",
"key": ""
}
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Ideally, the metrics generator would ignore invalid label names which will be rejected by prometheus.
Environment:
Additional Context