ES|QL pattern formatting by jgowdyelastic · Pull Request #222871 · elastic/kibana

jgowdyelastic · 2025-06-05T17:46:34Z

Adds a recommended query for the CATEGORIZE function in ES|QL.
Adds keyword highlighting for the patterns and the ability to open a new Discover tab to filter for docs which match the selected pattern.

2025-06-19.16-55-20.2025-06-19.16_55_49.mp4

…bana into esql-pattern-formatting

...iscover/public/context_awareness/profile_providers/common/patterns/pattern_cell_renderer.tsx

ThomThomson

Unified search changes LGTM! Code only review

joana-cps

The View matching results is also available in the count cell and it fails when I click on it. I think we should have this button in the pattern cell right?

@joana-cps Fixed in 4654f16

joana-cps

Should we limit the size of the popover? Looks a bit too much

@joana-cps I've added a maxWidth of 600px in b82feff

joana-cps

When I change the display to one line, I can't see the ... at the end and the tokens get cut off.

It works well with messages:

@joana-cps Fixed in 892daf8

joana-cps

Design changes LGTM ✅
Left some comments that can be addressed in a different PR as bug fixes

qn895 · 2025-06-24T16:25:09Z

src/platform/packages/shared/kbn-aiops-utils/src/get_categorization_field.ts

+
+const FIELD_PRIORITY = ['message', 'error.message', 'event.original'];
+
+const METADATA_FIELDS = [


Why not make this a set? So the METADATA_FIELDS array doesn't have be scanned every time we are checking the fields.

Yeah, good suggestion.

qn895 · 2025-06-24T16:30:54Z

x-pack/test/functional/apps/aiops/log_pattern_analysis_esql_in_discover.ts

+      const buttons = await popover.findAllByTagName('button');
+      expect(buttons.length).to.eql(4);
+      // Click the last button which is the "View docs in Discover" button
+      await buttons[3].click();


For future follow up: Is there a way to grab this button by ID instead of the order? There's a chance there will be more options added in the future in this popover, so suggestion to find maybe by ID or something more unique.

The data-test-subj has an ID appended to it, I couldn't work out how to know what this will be, so I went with the count of buttons instead.

We use this method in the dataGrid FTR service, which ignores the appended ID:

kibana/src/platform/test/functional/services/data_grid.ts

Lines 246 to 248 in dd29b09

public async clickCellExpandPopoverAction(cellActionId: string) {

await this.testSubjects.click(`*dataGridColumnCellAction-${cellActionId}`);

}

Changed in e697375

qn895

Left few comments for future follow up but LGTM 🎉

davismcphee

Just leaving a couple more quick comments, but again nothing that should block merging this.

...iscover/public/context_awareness/profile_providers/common/patterns/pattern_cell_renderer.tsx

davismcphee · 2025-06-25T00:27:55Z

...lugins/shared/discover/public/context_awareness/profile_providers/common/patterns/profile.ts

+            const discoverLocator = services.share?.url.locators.get('DISCOVER_APP_LOCATOR');
+            const discoverLink = discoverLocator?.getRedirectUrl({
+              query,
+              timeRange: services.data.query.timefilter.timefilter.getTime(),


Oh interesting, we'll have to fix that on our end then. In that case you're good to keep using timefilter if you'd like and we can adjust it later. It has no impact for now anyway, just hoping to at some point migrate away from the global services for some future plans we have.

davismcphee · 2025-06-25T00:29:28Z

x-pack/test/functional/apps/aiops/log_pattern_analysis_esql_in_discover.ts

+      const buttons = await popover.findAllByTagName('button');
+      expect(buttons.length).to.eql(4);
+      // Click the last button which is the "View docs in Discover" button
+      await buttons[3].click();


We use this method in the dataGrid FTR service, which ignores the appended ID:

kibana/src/platform/test/functional/services/data_grid.ts

Lines 246 to 248 in dd29b09

public async clickCellExpandPopoverAction(cellActionId: string) {

await this.testSubjects.click(`*dataGridColumnCellAction-${cellActionId}`);

}

...lugins/shared/discover/public/context_awareness/profile_providers/common/patterns/profile.ts

stratoula

This looks great, I added 2 additional comments that I prefer to be handled in this PR. They should be super small. I will approve immediately after!

.../shared/kbn-esql-validation-autocomplete/src/autocomplete/recommended_queries/suggestions.ts

...lugins/shared/discover/public/context_awareness/profile_providers/common/patterns/profile.ts

stratoula

ES|QL code changes LGTM! I also tested it locally and works great 🥳 Thanx James!

kibanamachine · 2025-06-25T10:33:51Z

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/15874029216

elasticmachine · 2025-06-25T10:35:35Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 5fdd09d

Failed CI Steps

FTR Configs #110

Test Failures

[job] [logs] FTR Configs #110 / Cloud Security Posture Test adding Cloud Security Posture Integrations CSPM AWS CIS_AWS Organization Manual Direct Access CIS_AWS Organization Manual Direct Access Workflow

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`aiops`	610	612	+2
`discover`	1333	1337	+4
`unifiedSearch`	369	371	+2
total			+8

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/esql-validation-autocomplete`	168	169	+1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`aiops`	509.3KB	509.0KB	-289.0B
`dataVisualizer`	681.3KB	680.0KB	-1.3KB
`discover`	1.1MB	1.1MB	+6.2KB
`unifiedSearch`	335.1KB	336.1KB	+983.0B
total			+5.5KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`dataVisualizer`	26.2KB	26.3KB	+31.0B
`discover`	21.9KB	21.9KB	+3.0B
`kbnUiSharedDeps-srcJs`	3.7MB	3.7MB	+772.0B
total			+806.0B

Unknown metric groups

API count

id	before	after	diff
`@kbn/aiops-utils`	-	4	+4
`@kbn/esql-utils`	114	116	+2
`@kbn/esql-validation-autocomplete`	191	192	+1
total			+7

async chunk count

id	before	after	diff
`dataVisualizer`	19	20	+1

References to deprecated APIs

id	before	after	diff
`@kbn/esql-utils`	30	31	+1

History

💚 Build #312172 succeeded 892daf8
💔 Build #312010 failed 09d65db
💔 Build #311868 failed c7bfab7
💔 Build #311664 failed 27043bd
💔 Build #311563 failed 53444d4
💔 Build #311431 failed 58ec14b

cc @jgowdyelastic

kibanamachine · 2025-06-25T10:40:56Z

💔 All backports failed

Status	Branch	Result
❌	8.19	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 222871

Questions ?

Please refer to the Backport tool documentation

jgowdyelastic · 2025-06-25T11:03:13Z

💚 All backports created successfully

Status	Branch	Result
✅	8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

Adds a recommended query for the `CATEGORIZE` function in ES|QL. Adds keyword highlighting for the patterns and the ability to open a new Discover tab to filter for docs which match the selected pattern. https://github.com/user-attachments/assets/9ed8c5b0-7e92-4cc8-88dd-cb7749b5ffd3 --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co> (cherry picked from commit 0d2930b) # Conflicts: # .github/CODEOWNERS # src/platform/packages/shared/kbn-esql-validation-autocomplete/src/autocomplete/recommended_queries/templates.ts # src/platform/plugins/shared/discover/public/application/main/hooks/use_esql_mode.test.tsx # src/platform/plugins/shared/discover/public/application/main/state_management/discover_app_state_container.ts # src/platform/plugins/shared/discover/public/application/main/state_management/discover_data_state_container.test.ts # src/platform/plugins/shared/discover/public/application/main/state_management/discover_data_state_container.ts # src/platform/plugins/shared/discover/public/application/main/state_management/utils/build_esql_fetch_subscribe.ts # src/platform/plugins/shared/discover/public/application/main/state_management/utils/change_data_view.ts

# Backport This will backport the following commits from `main` to `8.19`: - [ES|QL pattern formatting (#222871)](#222871)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>

jgowdyelastic and others added 29 commits June 5, 2025 18:46

ES|QL pattern formatting

98bb209

Merge branch 'main' into esql-pattern-formatting

6e3bd24

Merge branch 'main' into esql-pattern-formatting

f7c5aea

adding recommended query

c892304

Merge branch 'main' into esql-pattern-formatting

24d7481

Merge branch 'main' into esql-pattern-formatting

ed1f43e

fixing cell summary

4376d32

Merge branch 'main' into esql-pattern-formatting

f2d8941

Merge branch 'main' into esql-pattern-formatting

63117d3

adding link to show docs

b5e7db3

adjusting bottom margin

ff003fe

removing backslashes from keywords

4786ed8

fixing row height

2144a7e

Merge branch 'main' into esql-pattern-formatting

31f38b3

fixes after merge with main

5fb49c2

reverting auto change

cbdbfa3

moving function into package

4308e59

Merge branch 'main' into esql-pattern-formatting

8332328

[CI] Auto-commit changed files from 'node scripts/yarn_deduplicate'

e391a87

adding test

35c4b51

Merge branch 'esql-pattern-formatting' of github.com:jgowdyelastic/ki…

6840195

…bana into esql-pattern-formatting

Merge branch 'main' into esql-pattern-formatting

267ba71

moving package to avoid circular dependency

8825dfb

Merge branch 'esql-pattern-formatting' of github.com:jgowdyelastic/ki…

67d5a71

…bana into esql-pattern-formatting

Merge branch 'main' into esql-pattern-formatting

efb19ad

tests

a9d2e2d

fixing field selector and updating test

76b49cc

Merge branch 'main' into esql-pattern-formatting

3f7bf6e

Merge branch 'main' into esql-pattern-formatting

0b953c1

qn895 reviewed Jun 18, 2025

View reviewed changes

...iscover/public/context_awareness/profile_providers/common/patterns/pattern_cell_renderer.tsx Outdated Show resolved Hide resolved

renaming query title, setting default Count width

09d65db

ThomThomson approved these changes Jun 24, 2025

View reviewed changes

joana-cps reviewed Jun 24, 2025

View reviewed changes

fixing regex

faee0ee

joana-cps reviewed Jun 24, 2025

View reviewed changes

jgowdyelastic requested a review from joana-cps June 24, 2025 14:29

joana-cps approved these changes Jun 24, 2025

View reviewed changes

jgowdyelastic added 3 commits June 24, 2025 15:51

fixing count action and adding 600px min width to popover

4654f16

increasing maxWidth

b82feff

fixing row height is 1

892daf8

qn895 reviewed Jun 24, 2025

View reviewed changes

qn895 approved these changes Jun 24, 2025

View reviewed changes

davismcphee reviewed Jun 25, 2025

View reviewed changes

stratoula reviewed Jun 25, 2025

View reviewed changes

.../shared/kbn-esql-validation-autocomplete/src/autocomplete/recommended_queries/suggestions.ts Outdated Show resolved Hide resolved

...lugins/shared/discover/public/context_awareness/profile_providers/common/patterns/profile.ts Outdated Show resolved Hide resolved

changes based on review

e697375

stratoula approved these changes Jun 25, 2025

View reviewed changes

Merge branch 'main' into esql-pattern-formatting

5fdd09d

jgowdyelastic enabled auto-merge (squash) June 25, 2025 09:09

jgowdyelastic merged commit 0d2930b into elastic:main Jun 25, 2025
13 checks passed

jgowdyelastic mentioned this pull request Jun 25, 2025

[8.19] ES|QL pattern formatting (#222871) #225257

Merged

kibanamachine mentioned this pull request Jun 25, 2025

Adds missing model Claude 3.7 to accepted models #224943

Merged

2 tasks

peteharverson mentioned this pull request Jun 30, 2025

[ES|QL] CATEGORIZE / Pattern Analysis UX #215155

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ES|QL pattern formatting#222871

ES|QL pattern formatting#222871
jgowdyelastic merged 67 commits intoelastic:mainfrom
jgowdyelastic:esql-pattern-formatting

jgowdyelastic commented Jun 5, 2025 •

edited by kibanamachine

Loading

Uh oh!

ThomThomson left a comment

joana-cps left a comment •

edited by jgowdyelastic

Loading

joana-cps left a comment •

edited by jgowdyelastic

Loading

joana-cps left a comment •

edited by jgowdyelastic

Loading

joana-cps left a comment

qn895 Jun 24, 2025 •

edited

Loading

jgowdyelastic Jun 24, 2025

qn895 Jun 24, 2025

jgowdyelastic Jun 24, 2025

davismcphee Jun 25, 2025

jgowdyelastic Jun 25, 2025

qn895 left a comment

davismcphee left a comment

Uh oh!

davismcphee Jun 25, 2025

davismcphee Jun 25, 2025

Uh oh!

stratoula left a comment

Uh oh!

Uh oh!

stratoula left a comment

Uh oh!

kibanamachine commented Jun 25, 2025

elasticmachine commented Jun 25, 2025

API count

async chunk count

References to deprecated APIs

kibanamachine commented Jun 25, 2025

jgowdyelastic commented Jun 25, 2025

Labels

10 participants


		const FIELD_PRIORITY = ['message', 'error.message', 'event.original'];

		const METADATA_FIELDS = [

	public async clickCellExpandPopoverAction(cellActionId: string) {
	await this.testSubjects.click(`*dataGridColumnCellAction-${cellActionId}`);
	}

Conversation

jgowdyelastic commented Jun 5, 2025 • edited by kibanamachine Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ThomThomson left a comment

Choose a reason for hiding this comment

joana-cps left a comment • edited by jgowdyelastic Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

joana-cps left a comment • edited by jgowdyelastic Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

joana-cps left a comment • edited by jgowdyelastic Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

joana-cps left a comment

Choose a reason for hiding this comment

qn895 Jun 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

jgowdyelastic Jun 24, 2025

Choose a reason for hiding this comment

qn895 Jun 24, 2025

Choose a reason for hiding this comment

jgowdyelastic Jun 24, 2025

Choose a reason for hiding this comment

davismcphee Jun 25, 2025

Choose a reason for hiding this comment

jgowdyelastic Jun 25, 2025

Choose a reason for hiding this comment

qn895 left a comment

Choose a reason for hiding this comment

davismcphee left a comment

Choose a reason for hiding this comment

Uh oh!

davismcphee Jun 25, 2025

Choose a reason for hiding this comment

davismcphee Jun 25, 2025

Choose a reason for hiding this comment

Uh oh!

stratoula left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

stratoula left a comment

Choose a reason for hiding this comment

Uh oh!

kibanamachine commented Jun 25, 2025

elasticmachine commented Jun 25, 2025

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Page load bundle

API count

async chunk count

References to deprecated APIs

History

kibanamachine commented Jun 25, 2025

💔 All backports failed

Manual backport

Questions ?

jgowdyelastic commented Jun 25, 2025

💚 All backports created successfully

Questions ?

Labels

10 participants

jgowdyelastic commented Jun 5, 2025 •

edited by kibanamachine

Loading

joana-cps left a comment •

edited by jgowdyelastic

Loading

joana-cps left a comment •

edited by jgowdyelastic

Loading

joana-cps left a comment •

edited by jgowdyelastic

Loading

qn895 Jun 24, 2025 •

edited

Loading