[ES|QL] Add context-aware suggestion ordering with categorization by bartoval · Pull Request #243312 · elastic/kibana

bartoval · 2025-11-18T08:42:01Z

Work in progress

Summary

#197113

Add priority rules by new suggestItem prop called category.
rules take precedence and the lowest wins
You can increase or decrease the priority of certain rules on a per-command basis.
the final priority is the base priority + boost commands priority
monaco editor receives a new sequence of numbers normalized and based on the priority roles system (no more A, B, 1C..)

Notes:

We don't currently prioritize commands by preference (I don't know if it's needed, but it's easily extendable in the system)
The system is static for now

bartoval · 2025-11-18T08:44:12Z

src/platform/packages/shared/kbn-esql-ast/src/sorting/utils/operator_categories.ts

+ * License v3.0 only", or the "Server Side Public License, v 1".
+ */
+
+import {


I’m not sure we want this level of granularity for operators

I dont think we do either

bartoval · 2025-11-18T08:46:13Z

src/platform/packages/shared/kbn-esql-ast/src/sorting/priority_provider.ts

+
+// Context-specific priority adjustments (negative = boost up, positive = push down)
+const CONTEXT_BOOSTS: Record<string, Partial<Record<SuggestionCategory, number>>> = {
+  STATS: {


These are just examples to show how to change priorities per command

…ation

elasticmachine · 2025-11-20T07:59:45Z

Pinging @elastic/kibana-esql (Team:ESQL)

stratoula · 2025-11-20T09:46:28Z

@bartoval can you use enum for the categories and reuse it everywhere?

bartoval · 2025-11-20T14:34:26Z

@bartoval can you use enum for the categories and reuse it everywhere?

e6dec13

bartoval · 2025-11-21T08:22:25Z

src/platform/packages/shared/kbn-esql-ast/src/sorting/priority_provider.ts

+// Structure: { 'COMMAND' } or { 'COMMAND:LOCATION' } for more specific contexts
+const CONTEXT_BOOSTS: Record<string, Partial<Record<SuggestionCategory, number>>> = {
+  'STATS:BY': {
+    [SuggestionCategory.USER_DEFINED_COLUMN]: -300, // From 300 to 0


We have an integration test that does this only inside stats by: select and then double-click arrow down and expect to find create control after add histogram and col0 (🫤) . So I added this ability to do nested context sorting (which is nice anyway)

stratoula

This is a very nice enhancement, I just encountered some bugs and some feedback on the categories names to be easier to understand

src/platform/packages/shared/kbn-esql-ast/src/commands_registry/commands/enrich/util.ts

src/platform/packages/shared/kbn-esql-ast/src/commands_registry/complete_items.ts

stratoula · 2025-11-21T08:17:16Z

...form/packages/shared/kbn-esql-ast/src/commands_registry/options/recommended_queries/index.ts

  categorizationField?: string;
 }) => {
-  const queries = [
+  const queries: QueryTemplate[] = [


These will always be SuggestionCategory.RECOMMENDED_QUERY_SEARCH, why don't you put this in the helpers who give the suggestions?

getRecommendedQueriesTemplatesFromExtensions and getRecommendedQueriesSuggestionsFromStaticTemplates

but now we depends on Sortext in getRecommendedQueriesSuggestionsFromStaticTemplates. I thought we were going to deprecate it. The old code did sortText: query?.sortText ?? 'E',

sortText: query?.sortText ?? 'E', category: query.sortText === 'D' ? SuggestionCategory.RECOMMENDED_QUERY_SEARCH : SuggestionCategory.RECOMMENDED_QUERY,

Hmmm why do you need to do this?

Maybe I am not following but you have 2 types of queries

The ones that come from extensions --> getRecommendedQueriesTemplatesFromExtensions

The static ones --> getRecommendedQueriesSuggestionsFromStaticTemplates

The first have more priority

So apply the category there. why do you need to do this?

query.sortText === 'D' ? SuggestionCategory.RECOMMENDED_QUERY_SEARCH : SuggestionCategory.RECOMMENDED_QUERY,

src/platform/packages/shared/kbn-esql-ast/src/definitions/utils/functions.ts

src/platform/packages/shared/kbn-esql-ast/src/sorting/priority_provider.ts

src/platform/packages/shared/kbn-esql-ast/src/sorting/suggestion_ordering_engine.ts

src/platform/packages/shared/kbn-esql-ast/index.ts

stratoula · 2025-11-21T08:34:49Z

src/platform/packages/shared/kbn-esql-ast/index.ts

The aggregations have the highest priority in STATS, this should not change

Also in TS we want to prioritize the timeseries_aggs

The first has been fixed but not the second

stratoula · 2025-11-24T08:32:50Z

@bartoval can we make sure also that the select from the timepicker is second here??

Maybe it is already with your changes I didnt test

bartoval · 2025-11-24T09:10:57Z

@bartoval can we make sure also that the select from the timepicker is second here??
Maybe it is already with your changes I didnt test

Second place or is first place also okay?

If I move "choose from the time picker" to the CUSTOM ACTION group (same as create control), it will appear as the first item (alphabetical order with the same priority).
Otherwise I create a new CUSTOM_TIME_ACTION group and make it appear as the second.

stratoula · 2025-11-24T09:19:59Z

Second place or is first place also okay?

If I move "choose from the time picker" to the CUSTOM ACTION group (same as create control), it will appear as the first item (alphabetical order with the same priority).

Otherwise I create a new CUSTOM_TIME_ACTION group and make it appear as the second.

Be the first is also ok 🆗

stratoula

Looks much better, some small comments but we are very very close!!

stratoula · 2025-11-24T10:03:28Z

src/platform/packages/shared/kbn-esql-ast/index.ts

The first has been fixed but not the second

stratoula · 2025-11-24T10:10:11Z

...form/packages/shared/kbn-esql-ast/src/commands_registry/options/recommended_queries/index.ts

  categorizationField?: string;
 }) => {
-  const queries = [
+  const queries: QueryTemplate[] = [


Hmmm why do you need to do this?

Maybe I am not following but you have 2 types of queries

The ones that come from extensions --> getRecommendedQueriesTemplatesFromExtensions

The static ones --> getRecommendedQueriesSuggestionsFromStaticTemplates

The first have more priority

So apply the category there. why do you need to do this?

query.sortText === 'D' ? SuggestionCategory.RECOMMENDED_QUERY_SEARCH : SuggestionCategory.RECOMMENDED_QUERY,

stratoula · 2025-11-24T10:10:28Z

...form/packages/shared/kbn-esql-ast/src/commands_registry/options/recommended_queries/index.ts

      kind: 'Issue',
      detail: query.description,
      sortText: query?.sortText ?? 'E',
+      category:


I dont get why you have to do this 🤔 Check my other comment

Are you saying that the category field should not be here or or should not have the logic
query.sortText === 'D' ? SuggestionCategory.RECOMMENDED_QUERY_SEARCH : SuggestionCategory.RECOMMENDED_QUERY,

regarding this last point:
I did a different division. I considered all the queries as RECOMMENDED_QUERY and then just one RECOMMENDED_QUERY_SEARCH. Simply to respect the sort order. sortText: query?.sortText ?? 'E'

There is a query that must be at the top search all fields" suggestions. If it has the same priority as the others, it is sorted alphabetically.

What I don't understand is whether we want to deprecate sortText. If yes I apply another way to do controlled sorts within the same category, otherwise I'll use it.

ok I think I get it now. What we want is:

I did a different division. I considered all the queries as RECOMMENDED_QUERY and then just one RECOMMENDED_QUERY_SEARCH. Simply to respect the sort order.

Yes I get it, you are prioritizing the search query correctly, so this makes sense to have a different category. Can you just name it RECOMMENDED_QUERY_WITH_PRIORITY to be more generic? We might want to prioritize something else or more in the future

What I don't understand is whether we want to deprecate sortText

Yes we want. I am sorry I got confused I guess. We don't want this

query.sortText === 'D' ? SuggestionCategory.RECOMMENDED_QUERY_SEARCH : SuggestionCategory.RECOMMENDED_QUERY

your previous implementation makes pretty sense now, sorryyyyy

...form/packages/shared/kbn-esql-ast/src/commands_registry/options/recommended_queries/index.ts

bartoval · 2025-11-24T12:40:42Z

Looks much better, some small comments but we are very very close!!

Second place or is first place also okay?

If I move "choose from the time picker" to the CUSTOM ACTION group (same as create control), it will appear as the first item (alphabetical order with the same priority).

Otherwise I create a new CUSTOM_TIME_ACTION group and make it appear as the second.

Be the first is also ok 🆗

now an integration test in discover fails. There are tests that expect some suggestion in fixed positions. so I apply the other way ... or I go to change the test

stratoula · 2025-11-24T13:02:11Z

now an integration test in discover fails. There are tests that expect some suggestion in fixed positions. so I apply the other way ... or I go to change the test

Change the test!

elasticmachine · 2025-11-24T13:48:33Z

💔 Build Failed

Buildkite Build
Commit: 9e90fe9

Failed CI Steps

Check Types

History

💔 Build #364183 failed 1f5ec49
💚 Build #364098 succeeded c39f7bd
💛 Build #363832 was flaky 74b85ab
💚 Build #363547 succeeded 5eac409
💚 Build #363406 succeeded 75a9ad2

cc @bartoval

stratoula

Yesss! It looks really nice 👏

…astic#243312) **Work in progress** ## Summary elastic#197113 - Add priority rules by new suggestItem prop called category. - rules take precedence and the lowest wins - You can increase or decrease the priority of certain rules on a per-command basis. - the final priority is the base priority + boost commands priority - monaco editor receives a **new sequence of numbers normalized** and based on the priority roles system (no more A, B, 1C..) **Notes:** - We don't currently prioritize commands by preference (I don't know if it's needed, but it's easily extendable in the system) - The system is static for now --------- Co-authored-by: Stratou <efstratia.kalafateli@elastic.co>

bartoval self-assigned this Nov 18, 2025

bartoval requested a review from a team as a code owner November 18, 2025 08:42

bartoval marked this pull request as draft November 18, 2025 08:42

bartoval added the WIP Work in progress label Nov 18, 2025

bartoval commented Nov 18, 2025

View reviewed changes

bartoval force-pushed the strong_suggestions_priority branch 5 times, most recently from 7eb7f7b to 3dae845 Compare November 19, 2025 18:49

bartoval added 2 commits November 19, 2025 19:57

[ES|QL]

724d9e0

[ES|QL] Add context-aware suggestion ordering with operator categoriz…

5b9fe49

…ation

bartoval force-pushed the strong_suggestions_priority branch 4 times, most recently from 768af6f to f0732b3 Compare November 20, 2025 00:20

solution without sortext

2475929

bartoval force-pushed the strong_suggestions_priority branch from f0732b3 to 2475929 Compare November 20, 2025 07:52

bartoval changed the title ~~[ES|QL] Add context-aware suggestion ordering with categorization (WIP)~~ Nov 20, 2025

bartoval added release_note:enhancement backport:skip This PR does not require backporting Feature:ES|QL ES|QL related features in Kibana Team:ESQL ES|QL related features in Kibana t// v9.3.0 and removed WIP Work in progress labels Nov 20, 2025

bartoval marked this pull request as ready for review November 20, 2025 07:59

Merge branch 'main' into strong_suggestions_priority

9f81fb4

bartoval force-pushed the strong_suggestions_priority branch from 124ff5e to 9f81fb4 Compare November 20, 2025 08:05

bartoval force-pushed the strong_suggestions_priority branch from 4059e13 to e6dec13 Compare November 20, 2025 14:31

update from feedbacks

75a9ad2

bartoval force-pushed the strong_suggestions_priority branch from e6dec13 to 75a9ad2 Compare November 20, 2025 20:09

Merge branch 'main' into strong_suggestions_priority

5eac409

bartoval commented Nov 21, 2025

View reviewed changes

stratoula reviewed Nov 21, 2025

View reviewed changes

bartoval and others added 2 commits November 21, 2025 16:55

update from comments

74b85ab

Merge branch 'main' into strong_suggestions_priority

c39f7bd

bartoval requested a review from stratoula November 24, 2025 06:43

update feedback

cc0e113

stratoula reviewed Nov 24, 2025

View reviewed changes

Merge branch 'main' into strong_suggestions_priority

34f43bc

bartoval force-pushed the strong_suggestions_priority branch from 1f5ec49 to 9e90fe9 Compare November 24, 2025 13:38

change integration tests

5708c8c

bartoval force-pushed the strong_suggestions_priority branch from 9e90fe9 to 5708c8c Compare November 24, 2025 14:54

change ts function priority for stats

79bb20d

stratoula approved these changes Nov 24, 2025

View reviewed changes

bartoval merged commit 2f93b87 into elastic:main Nov 25, 2025
12 checks passed

Conversation

bartoval commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

bartoval Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elasticmachine commented Nov 20, 2025

stratoula commented Nov 20, 2025

bartoval commented Nov 20, 2025

bartoval Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

stratoula left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

bartoval Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stratoula commented Nov 24, 2025

bartoval commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

stratoula commented Nov 24, 2025

stratoula left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bartoval Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

stratoula Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bartoval commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

stratoula commented Nov 24, 2025

elasticmachine commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💔 Build Failed

Failed CI Steps

History

stratoula left a comment

Choose a reason for hiding this comment

Uh oh!

Labels

3 participants

bartoval commented Nov 18, 2025 •

edited

Loading

bartoval Nov 18, 2025 •

edited

Loading

bartoval Nov 21, 2025 •

edited

Loading

bartoval Nov 21, 2025 •

edited

Loading

bartoval commented Nov 24, 2025 •

edited

Loading

bartoval Nov 24, 2025 •

edited

Loading

stratoula Nov 24, 2025 •

edited

Loading

bartoval commented Nov 24, 2025 •

edited

Loading

elasticmachine commented Nov 24, 2025 •

edited

Loading