Skip to content

Bug: some metrics fail to follow Prometheus/OpenMetrics naming requirements #12635

@krajorama

Description

@krajorama

What is the bug?

Checking metrics by running local dev:

curl -v -H "Accept: application/openmetrics-text" http://localhost:8101/metrics |grep -E "^# TYPE.* unknown$"

# TYPE cortex_bucket_store_cached_postings_compression_time_seconds unknown
# TYPE cortex_mimir_query_engine_common_subexpression_elimination_duplication_nodes_introduced unknown
# TYPE cortex_mimir_query_engine_common_subexpression_elimination_selectors_eliminated unknown
# TYPE cortex_mimir_query_engine_common_subexpression_elimination_selectors_inspected unknown
# TYPE cortex_query_frontend_regexp_matcher_count unknown
# TYPE cortex_query_frontend_regexp_matcher_optimized_count unknown

These are counters, but do not follow the standard naming and lack _total suffix, which is why they are not recognized as counters.

There might be more.

This is not great for UIs or sending over remote write to OTEL collector, because that complains:

time=2025-09-07T13:15:11.241+02:00 level=ERROR source=queue_manager.go:1689 msg="non-recoverable error" component=remote remote_name=f8e8af url=http://localhost:4318/api/v1/write failedSampleCount=117 failedHistogramCount=13 failedExemplarCount=0 err="server returned HTTP status 400 Bad Request: unsupported metric type \"METRIC_TYPE_UNSPECIFIED\" for metric \"cortex_bucket_store_cached_postings_compression_time_seconds\"\nunsupported metric type \"METRIC_TYPE_UNSPECIFIED\" for metric \"cortex_bucket_store_cached_postings_compression_time_seconds\"\nunsupported metric type \"METRIC_TYPE_UNSPECIFIED\" for metric \"cortex_mimir_query_engine_common_subexpression_elimination_duplication_nodes_introduced\"\nunsupported metric type \"METRIC_TYPE_UNSPECIFIED\" for metric \"cortex_mimir_query_engine_common_subexpression_elimination_selectors_eliminated\"\nunsupported metric type \"METRIC_TYPE_UNSPECIFIED\" for metric \"cortex_mimir_query_engine_common_subexpression_elimination_selectors_inspected\"\nunsupported metric type \"METRIC_TYPE_UNSPECIFIED\" for metric \"cortex_query_frontend_regexp_matcher_count\"\nunsupported metric type \"METRIC_TYPE_UNSPECIFIED\" for metric \"cortex_query_frontend_regexp_matcher_optimized_count\"\n"

How to reproduce it?

Start Mimir in local dev env and curl get metrics by way of OpenMetrics.

curl -v -H "Accept: application/openmetrics-text" http://localhost:8101/metrics |grep -E "^# TYPE.* unknown$"

What did you think would happen?

Mimir should not emit non conforming metrics.

What was your environment?

Local dev, linux docker.

Any additional context to share?

See prometheus/prometheus#17075 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/opentelemetryEverything related to OpenTelemetry OTLP OTelbugSomething isn't working

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions