-
Notifications
You must be signed in to change notification settings - Fork 658
Open
Labels
area/opentelemetryEverything related to OpenTelemetry OTLP OTelEverything related to OpenTelemetry OTLP OTelbugSomething isn't workingSomething isn't working
Description
What is the bug?
Checking metrics by running local dev:
curl -v -H "Accept: application/openmetrics-text" http://localhost:8101/metrics |grep -E "^# TYPE.* unknown$"
# TYPE cortex_bucket_store_cached_postings_compression_time_seconds unknown
# TYPE cortex_mimir_query_engine_common_subexpression_elimination_duplication_nodes_introduced unknown
# TYPE cortex_mimir_query_engine_common_subexpression_elimination_selectors_eliminated unknown
# TYPE cortex_mimir_query_engine_common_subexpression_elimination_selectors_inspected unknown
# TYPE cortex_query_frontend_regexp_matcher_count unknown
# TYPE cortex_query_frontend_regexp_matcher_optimized_count unknown
These are counters, but do not follow the standard naming and lack _total suffix, which is why they are not recognized as counters.
There might be more.
This is not great for UIs or sending over remote write to OTEL collector, because that complains:
time=2025-09-07T13:15:11.241+02:00 level=ERROR source=queue_manager.go:1689 msg="non-recoverable error" component=remote remote_name=f8e8af url=http://localhost:4318/api/v1/write failedSampleCount=117 failedHistogramCount=13 failedExemplarCount=0 err="server returned HTTP status 400 Bad Request: unsupported metric type \"METRIC_TYPE_UNSPECIFIED\" for metric \"cortex_bucket_store_cached_postings_compression_time_seconds\"\nunsupported metric type \"METRIC_TYPE_UNSPECIFIED\" for metric \"cortex_bucket_store_cached_postings_compression_time_seconds\"\nunsupported metric type \"METRIC_TYPE_UNSPECIFIED\" for metric \"cortex_mimir_query_engine_common_subexpression_elimination_duplication_nodes_introduced\"\nunsupported metric type \"METRIC_TYPE_UNSPECIFIED\" for metric \"cortex_mimir_query_engine_common_subexpression_elimination_selectors_eliminated\"\nunsupported metric type \"METRIC_TYPE_UNSPECIFIED\" for metric \"cortex_mimir_query_engine_common_subexpression_elimination_selectors_inspected\"\nunsupported metric type \"METRIC_TYPE_UNSPECIFIED\" for metric \"cortex_query_frontend_regexp_matcher_count\"\nunsupported metric type \"METRIC_TYPE_UNSPECIFIED\" for metric \"cortex_query_frontend_regexp_matcher_optimized_count\"\n"
How to reproduce it?
Start Mimir in local dev env and curl get metrics by way of OpenMetrics.
curl -v -H "Accept: application/openmetrics-text" http://localhost:8101/metrics |grep -E "^# TYPE.* unknown$"
What did you think would happen?
Mimir should not emit non conforming metrics.
What was your environment?
Local dev, linux docker.
Any additional context to share?
Metadata
Metadata
Assignees
Labels
area/opentelemetryEverything related to OpenTelemetry OTLP OTelEverything related to OpenTelemetry OTLP OTelbugSomething isn't workingSomething isn't working