[ES|QL] Index editor - Allow selecting column type by sddonne · Pull Request #241637 · elastic/kibana

sddonne · 2025-11-03T14:42:44Z

Summary

Now it's possible to select the column type in the Lookup Index Editor, instead of relying in ES dynamic mapping.

Known Limitations

Field types that are not supported by ES|QL are allowed to be selected, and values can be pushed to the index, but are not going to be visible neither in Discover or if the flyout is reopened.

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

Unit or functional tests were updated or added to match the most common scenarios
Flaky Test Runner was used on any tests changed

…-mappings

AlexGPlay · 2025-11-24T09:02:06Z

src/platform/packages/shared/kbn-field-utils/src/components/field_select/field_select.test.tsx

+  });
+
+  it('should display placeholder when no type is selected', () => {
+    render(<FieldSelect selectedType={null} onTypeChange={mockOnTypeChange} />);


just in case it works better for you too, not a mandatory or anything more of a how i usually do tests - i usually like to have a set up method that does the rendering and creates the defaults, in this case it'd remove the need for the beforeEach and for the onTypeChange setting every time, something like:

const setup = (props: Partial<React.ComponentProps<typeof FieldSelect>> = {}) => { const mockOnTypeChange = jest.fn() const user = userEvent.setup() render(<FieldSelect selectedType={null} onTypeChange={mockOnTypeChange} {...props} />) return { mockOnTypeChange, user } } describe('FieldSelect', () => { ... it('should call onTypeChange with null when selection is cleared', async () => { const { user, mockOnTypeChange } = setup() const clearButton = screen.getByTestId('comboBoxClearButton'); await user.click(clearButton); expect(mockOnTypeChange).toHaveBeenCalledWith(null); }); ... }

AlexGPlay

code review only LGTM but not sure if anyone with more context wants to double check this

jughosta

Nice addition to the flow!

jughosta · 2025-11-25T13:07:02Z

src/platform/packages/shared/kbn-field-utils/src/components/field_select/utils.ts

+  {
+    label: i18n.translate('fieldUtils.fieldTypeSelect.group.booleanDate', {
+      defaultMessage: 'Boolean & Date',
+    }),
+    types: ['boolean', 'date', 'date_nanos'],
+  },
+  {
+    label: i18n.translate('fieldUtils.fieldTypeSelect.group.ipGeo', {
+      defaultMessage: 'IP & Geo',
+    }),
+    types: ['ip', 'geo_point', 'geo_shape'],
+  },


This grouping is a bit confusing. Can we have them separately?

Or it could be 3 groups: Common (with boolean, binary), Dates and Geo.
And additionally Structured (with ip, version). It would be closer to docs categories

Mmm you are right @jughosta, the groups can do better.

What do you think of this distribution? (I'm not using Common as I think it's not the best for grouping boolean and binary, if we use Common maybe it should contain all the most used types.)

@l-suarez whats your opinion on the groups?

Text

'text', 'keyword', 'constant_keyword', 'wildcard', 'match_only_text', 'search_as_you_type', 'semantic_text',

Numeric

'byte', 'double', 'float', 'half_float', 'integer', 'long', 'short', 'unsigned_long', 'histogram',

Dates

'date', 'date_nanos'

Boolean & Binary

'boolean', 'binary'

Geo

'geo_point', 'geo_shape'

Object

'flattened'

Structured

'ip', 'version', 'integer_range', 'float_range', 'long_range', 'ip_range', 'double_range', 'date_range',

Advanced

'rank_feature', 'rank_features', 'dense_vector', 'sparse_vector', 'percolator'

Screen.Recording.2025-11-25.at.15.03.23.mov

I think the groups are good as they are. We should not add "Common". We reference to the documentation from the current field type filter on DSL and here we also don't have "Common". The groups seems fine for me.

jughosta · 2025-11-25T13:12:59Z

src/platform/packages/shared/kbn-field-utils/src/components/field_select/utils.ts

+  },
+  {
+    label: i18n.translate('fieldUtils.fieldTypeSelect.group.structured', {
+      defaultMessage: 'Structured',


According to our docs it would be Object.

Suggested change

defaultMessage: 'Structured',

defaultMessage: 'Object',

jughosta · 2025-11-25T13:25:04Z

...ages/private/kbn-index-editor/src/components/grid_custom_renderers/column_header_popover.tsx

+                data-test-subj="indexEditorColumnNameInput"
+                value={columnName}
+                placeholder={i18n.translate('indexEditor.columnHeaderEdit.columnNamePlaceholder', {
+                  defaultMessage: 'Choose the name of the field',


Another option:

Suggested change

defaultMessage: 'Choose the name of the field',

defaultMessage: 'Enter field name',

Unless you've a strong position in this one I would like to stick with the design proposal https://www.figma.com/design/QltbaeErcZzbVbSqwpccvG/Lookup-UX?node-id=2003-74648&p=f&t=qS2l8kBpKAF2pM8c-0 🙏

Just looked at the designs - We should go with @jughosta's proposal if possible 🙏

Updated, thank you both! 9aa9600

jughosta · 2025-11-25T13:25:35Z

...ages/private/kbn-index-editor/src/components/grid_custom_renderers/column_header_popover.tsx

+            defaultMessage: 'Select a field type',
+          })}
+          helpText={i18n.translate('indexEditor.columnHeaderEdit.fieldTypeHelpText', {
+            defaultMessage: `You won't be able to change the type after saving it.`,


Suggested change

defaultMessage: `You won't be able to change the type after saving it.`,

defaultMessage: `You won't be able to change the type after saving the lookup index.`,

Updated! 98214fd

jughosta · 2025-11-25T13:27:12Z

...ages/private/kbn-index-editor/src/components/grid_custom_renderers/column_header_popover.tsx

+      <EuiForm component="form" onSubmit={onSubmit}>
+        <EuiFormRow
+          label={i18n.translate('indexEditor.columnHeaderEdit.fieldType', {
+            defaultMessage: 'Select a field type',


Suggested change

defaultMessage: 'Select a field type',

defaultMessage: 'Select field type',

Wonder if it would be correct this way for a label too.

Same with this one 🙏

elasticmachine · 2025-11-26T10:05:38Z

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)

…onne/kibana into index-editor-column-mappings

sddonne · 2025-11-26T11:56:32Z

src/platform/packages/shared/kbn-field-utils/src/components/field_select/field_select.tsx

Two changes based on @ninoslavmiskovic feedback:

Take out the box from the selected item

Don't hide the name input when clearing the field type.

f243793

jughosta

LGTM, thanks!

elasticmachine · 2025-11-27T11:46:14Z

💚 Build Succeeded

Buildkite Build
Commit: 9aa9600

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`aiops`	607	610	+3
`apm`	2045	2048	+3
`cloudSecurityPosture`	655	658	+3
`datasetQuality`	894	897	+3
`dataViewManagement`	183	186	+3
`dataVisualizer`	835	838	+3
`discover`	1845	1848	+3
`esql`	505	509	+4
`esqlDataGrid`	280	283	+3
`eventAnnotationListing`	580	583	+3
`lens`	1608	1611	+3
`maps`	1290	1293	+3
`ml`	2811	2814	+3
`observability`	1600	1603	+3
`presentationUtil`	105	108	+3
`securitySolution`	8431	8434	+3
`slo`	1314	1317	+3
`streamsApp`	1096	1099	+3
`unifiedDocViewer`	404	407	+3
`unifiedSearch`	532	535	+3
total			+61

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/esql-types`	73	74	+1
`@kbn/field-utils`	42	48	+6
total			+7

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`aiops`	527.4KB	527.6KB	+166.0B
`apm`	2.8MB	2.8MB	+166.0B
`cloudSecurityPosture`	642.7KB	643.0KB	+332.0B
`datasetQuality`	428.4KB	428.6KB	+166.0B
`dataViewManagement`	141.9KB	142.1KB	+166.0B
`dataVisualizer`	596.9KB	597.2KB	+332.0B
`discover`	1.2MB	1.2MB	+332.0B
`enterpriseSearch`	947.1KB	947.2KB	+166.0B
`esql`	668.2KB	669.7KB	+1.5KB
`esqlDataGrid`	149.9KB	150.2KB	+332.0B
`eventAnnotationListing`	206.9KB	207.0KB	+166.0B
`graph`	372.6KB	372.7KB	+166.0B
`indexManagement`	707.8KB	707.9KB	+166.0B
`lens`	1.6MB	1.6MB	+166.0B
`maps`	3.1MB	3.1MB	+166.0B
`ml`	5.6MB	5.6MB	+332.0B
`presentationUtil`	67.9KB	68.1KB	+166.0B
`securitySolution`	11.1MB	11.1MB	+166.0B
`slo`	988.3KB	988.7KB	+332.0B
`stackAlerts`	68.9KB	69.1KB	+166.0B
`streamsApp`	1.0MB	1.0MB	+166.0B
`transform`	624.9KB	625.1KB	+166.0B
`unifiedDocViewer`	276.0KB	276.2KB	+166.0B
`unifiedSearch`	393.0KB	393.2KB	+166.0B
`workflowsManagement`	2.1MB	2.1MB	+166.0B
total			+6.3KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`esql`	18.8KB	18.8KB	+34.0B
`osquery`	42.8KB	43.0KB	+166.0B
total			+200.0B

Unknown metric groups

API count

id	before	after	diff
`@kbn/esql-types`	76	77	+1
`@kbn/field-utils`	51	58	+7
total			+8

ESLint disabled line counts

id	before	after	diff
`@kbn/index-editor`	3	4	+1

Total ESLint disabled count

id	before	after	diff
`@kbn/index-editor`	3	4	+1

History

💚 Build #365829 succeeded 7864247
💔 Build #365728 failed 41e9f7a
💛 Build #365051 was flaky 98214fd
💛 Build #363798 was flaky 16dfb3a
💚 Build #363708 succeeded 2953d51
💔 Build #363241 failed 0140a05

cc @sddonne

## Summary Now it's possible to select the column type in the Lookup Index Editor, instead of relying in ES dynamic mapping. ![column-types](https://github.com/user-attachments/assets/74179fcc-3827-4800-b966-3e21b78d39af) ### Known Limitations Field types that are not supported by ES|QL are allowed to be selected, and values can be pushed to the index, but are not going to be visible neither in Discover or if the flyout is reopened. <img width="415" height="668" alt="image" src="https://github.com/user-attachments/assets/593f31ea-c29c-418b-a75e-f040cd91806d" /> ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Stratou <efstratia.kalafateli@elastic.co>

sddonne added 30 commits October 31, 2025 16:44

open popover on column click

4c0501f

add form UI

5f136a6

use popover footer

9b695fa

add validations

af9a302

keep fieldType state

9569212

display type icon after selecting it

3f879e3

don't use children prop

4136368

fix: values lost when changing type

f8d52c1

Add update mapping route

290aff9

save mappings before bulk update

f6fa672

Merge remote-tracking branch 'upstream/main' into index-editor-column…

3aad603

…-mappings

accessibility improvements

4014f0e

split in two files

78910d2

clean service

3cbf835

Merge remote-tracking branch 'upstream/main' into index-editor-column…

77d65a9

…-mappings

Add field types support

a93bc25

add more types

e08ae8e

Merge remote-tracking branch 'upstream/main' into index-editor-column…

6aa8c5b

…-mappings

Merge remote-tracking branch 'upstream/main' into index-editor-column…

616b8f2

…-mappings

use constant for mappings endpoint

b2e14ca

improve field selector code

fbfd7a3

display warning icon on unsupported field

63ef6a6

update wording

cdcb31c

move field select to package

1d9f774

fix types

821a23a

use esql async result to detect unsupported types

5cfec1e

update integration tests

91ef7d4

add missing tokens

5e381cd

fix token

11b1961

Merge remote-tracking branch 'upstream/main' into index-editor-column…

eda3ab1

…-mappings

AlexGPlay reviewed Nov 24, 2025

View reviewed changes

kertal requested a review from a team November 25, 2025 06:44

jughosta reviewed Nov 25, 2025

View reviewed changes

sddonne and others added 3 commits November 25, 2025 14:49

Update wordings

98214fd

Merge branch 'main' into index-editor-column-mappings

85a6fb6

Changes from node scripts/lint_ts_projects --fix

7db043d

elastic-vault-github-plugin-prod bot requested a review from a team as a code owner November 26, 2025 09:37

Changes from node scripts/regenerate_moon_projects.js --update

eddd6fc

elastic-vault-github-plugin-prod bot requested a review from a team as a code owner November 26, 2025 10:05

botelastic bot added the Team:obs-ux-management label Nov 26, 2025

sddonne added 3 commits November 26, 2025 12:36

Merge branch 'main' into index-editor-column-mappings

476f0e8

select UX improvement

f243793

Merge branch 'index-editor-column-mappings' of https://github.com/sdd…

41e9f7a

…onne/kibana into index-editor-column-mappings

sddonne commented Nov 26, 2025

View reviewed changes

darnautov removed the request for review from a team November 26, 2025 13:01

sddonne removed the Team:obs-ux-management label Nov 26, 2025

sddonne removed the request for review from a team November 26, 2025 14:01

fix test

7864247

sddonne requested a review from jughosta November 27, 2025 08:25

update wording

9aa9600

jughosta approved these changes Nov 27, 2025

View reviewed changes

sddonne enabled auto-merge (squash) November 27, 2025 10:03

sddonne merged commit a7f8bb6 into elastic:main Nov 27, 2025
12 checks passed

florent-leborgne added the needs_docs label Dec 10, 2025

florent-leborgne mentioned this pull request Dec 10, 2025

[ES|QL editor] Ability to select data type for lookup index columns elastic/docs-content#4277

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ES|QL] Index editor - Allow selecting column type#241637

[ES|QL] Index editor - Allow selecting column type#241637
sddonne merged 67 commits intoelastic:mainfrom
sddonne:index-editor-column-mappings

sddonne commented Nov 3, 2025 •

edited

Loading

AlexGPlay Nov 24, 2025

AlexGPlay left a comment

jughosta left a comment

jughosta Nov 25, 2025

jughosta Nov 25, 2025

sddonne Nov 25, 2025

ninoslavmiskovic Nov 26, 2025

jughosta Nov 25, 2025

sddonne Nov 25, 2025

jughosta Nov 25, 2025

sddonne Nov 27, 2025

florent-leborgne Nov 27, 2025

sddonne Nov 27, 2025

jughosta Nov 25, 2025

sddonne Nov 25, 2025

jughosta Nov 25, 2025

sddonne Nov 27, 2025

elasticmachine commented Nov 26, 2025

sddonne Nov 26, 2025 •

edited

Loading

jughosta left a comment

Uh oh!

elasticmachine commented Nov 27, 2025

API count

ESLint disabled line counts

Total ESLint disabled count

Labels

8 participants

	defaultMessage: 'Choose the name of the field',
	defaultMessage: 'Enter field name',

	defaultMessage: `You won't be able to change the type after saving it.`,
	defaultMessage: `You won't be able to change the type after saving the lookup index.`,

	defaultMessage: 'Select a field type',
	defaultMessage: 'Select field type',

Conversation

sddonne commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Known Limitations

Checklist

Choose a reason for hiding this comment

AlexGPlay left a comment

Choose a reason for hiding this comment

jughosta left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Text

Numeric

Dates

Boolean & Binary

Geo

Object

Structured

Advanced

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elasticmachine commented Nov 26, 2025

sddonne Nov 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

jughosta left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Nov 27, 2025

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Page load bundle

API count

ESLint disabled line counts

Total ESLint disabled count

History

Labels

8 participants

sddonne commented Nov 3, 2025 •

edited

Loading

sddonne Nov 26, 2025 •

edited

Loading