[ES|QL] Support new exponential_histogram ES Field type#242748
[ES|QL] Support new exponential_histogram ES Field type#242748bartoval merged 7 commits intoelastic:mainfrom
Conversation
bartoval
added
release_note:fix
backport:skip
|
Pinging @elastic/kibana-esql (Team:ESQL) |
|
@jughosta Tagging you for the review on this as the most familiar on our end with the field types code. Also a heads up @drewdaemon about bundle size increases, which seem fairly significant for such a small change. Maybe this is a case where we'd benefit from adding the fields packages to our shared deps bundle so we only take the hit once? Didn't look into it though, just for awareness. |
bartoval
force-pushed
the
support_esql_extended_histogram
branch
from
f5149a5 to
5b4d82d
Compare
stratoula
left a comment
stratoula
left a comment
There was a problem hiding this comment.
Shouldn't we added this also here? src/platform/packages/shared/kbn-esql-ast/src/definitions/types.ts
In order when the definitions get updated and the function support this field type to have it ready (check the type in the signatures, here for example src/platform/packages/shared/kbn-esql-ast/src/definitions/generated/aggregation_functions.ts)
src/platform/packages/shared/kbn-field-types/src/kbn_field_types_factory.ts
Outdated
Show resolved
Hide resolved
bartoval
force-pushed
the
support_esql_extended_histogram
branch
from
5b4d82d to
5b6ac96
Compare
| rank_features: { iconType: 'tokenRankFeatures' }, | ||
| histogram: { iconType: 'tokenHistogram' }, | ||
| exponential_histogram: { iconType: 'tokenHistogram' }, | ||
| tdigest: { iconType: 'tokenHistogram' }, |
There was a problem hiding this comment.
???
I don't see any other options
There was a problem hiding this comment.
I think it is ok to use the same. These 2 field types are connected with histograms. ES is going to add support for the existing histogram type in ESQL by casting to one of these
|
it’s quicker to ask here.... The tdigest follows all the same steps as exponential_histogram. But here I don’t see any icons for response_time , and what I see in discover is null. PUT /test-tdigest
{
"mappings": {
"properties": {
"@timestamp": {
"type": "date"
},
"response_time": {
"type": "tdigest"
}
}
}
}POST /test-tdigest/_doc
{
"@timestamp": "2025-11-13T10:00:00Z",
"response_time": {
"centroids": [2.0, 3.4],
"counts": [1, 4]
}
}also should the info be added here too? |
|
Do you mean that when you run this index in discovr, you dont see this column?? If yes, then this is possibly happening because they don't support this part yet. It is ok though I think, we can add support at kibana |
yes I think so. Elastic side data exist 
but these values exists
|
stratoula
left a comment
stratoula
left a comment
There was a problem hiding this comment.
I didnt test again, code review only, LGTM!
jughosta
left a comment
jughosta
left a comment
There was a problem hiding this comment.
LGTM 👍
exponential_histogram still shows up in the breakdown dropdown though. Maybe it's a bug in the utils we have for filtering it.
|
How about we modify the following utils to exclude these new types?
|
Is that a bad idea ? (in esql_fields_utils.ts) in the function import { getKbnFieldType } from '@kbn/field-types';
....
if (type) {
const kbnFieldType = getKbnFieldType(type);
if (!kbnFieldType.filterable) {
return false;
} |
|
Hm, not sure about it. If we had Also, I can't remember why |
it can be a check also in is own parent These are just ideas to avoid putting TODO and then manually removing them in the future (if I understand correctly this breackdown button should be disabled for the condition filterable false as a role) |
|
ok @jughosta I applied your suggestions, Thanks! |
bartoval
force-pushed
the
support_esql_extended_histogram
branch
from
98d296a to
86381e6
Compare
The BY operator in ES|QL is not an aggregation. In DSL you are applying the terms aggregation to create the breakdown, this is why you are checking if the field is aggregateble or not. In ES|QL you cant have the same checks. The BY operator allows more capabilities and fields to create the breakdown and this is why we have different flags |
This is not great I agree, but you can't use the isFilterable either. Neither the iisAggregatabler though (this is a DSL thingy). One field type can be used in where but is not supported in the BY. We could create an |
💚 Build Succeeded
Metrics [docs]Async chunks
Page load bundle
History
cc @bartoval |
|
Thanks for updating the utils! |
## Summary https://github.com/elastic/elasticsearch/blob/main/x-pack/plugin/esql/qa/testFixtures/src/main/resources/data/exp_histo_sample.csv elastic#242699 From the dev tool ```bash PUT test-histogram { "mappings": { "properties": { "@timestamp": {"type": "date"}, "response_time": {"type": "exponential_histogram"} } } } ``` ```bash POST /test-histogram/_doc { "@timestamp": "2025-11-12T17:00:00Z", "response_time": { "scale": 0, "positive": { "indices": [0, 1, 2], "counts": [5, 10, 3] } } } ``` <img width="1060" height="484" alt="screen" src="https://github.com/user-attachments/assets/a20992ee-c089-4ea2-b3c4-ea93f16adadd" />
6 participants
Summary
https://github.com/elastic/elasticsearch/blob/main/x-pack/plugin/esql/qa/testFixtures/src/main/resources/data/exp_histo_sample.csv
#242699
From the dev tool
PUT test-histogram { "mappings": { "properties": { "@timestamp": {"type": "date"}, "response_time": {"type": "exponential_histogram"} } } }POST /test-histogram/_doc { "@timestamp": "2025-11-12T17:00:00Z", "response_time": { "scale": 0, "positive": { "indices": [0, 1, 2], "counts": [5, 10, 3] } } }