Skip to content

[ES|QL] improve function validation#230139

Merged
drewdaemon merged 111 commits intoelastic:mainfrom
drewdaemon:221230/change-function-validation-scheme
Aug 28, 2025
Merged

[ES|QL] improve function validation#230139
drewdaemon merged 111 commits intoelastic:mainfrom
drewdaemon:221230/change-function-validation-scheme

Conversation

@drewdaemon
Copy link
Contributor

@drewdaemon drewdaemon commented Jul 31, 2025

Summary

Closes #180518

Part of #221230

This is a full rewrite of our function validation logic. The primary goal is to simplify our code, though the algorithm has also been simplified and some user-facing validation messages have been changed.

For reviewers

I know... it looks big 😞 But most of it is updating tests! ✨

The place to start is src/platform/packages/shared/kbn-esql-ast/src/definitions/utils/validation/function.ts. All the other changes stem from this rewrite.

I did come across test suites that weren't worth updating and I removed those. I've tried to leave comments where I felt that was the case. I did significantly extend our test coverage in the functions.test.ts file where most of our function tests should live.

I tried to be careful, but I wouldn't be surprised if there are some regressions. I have included a list of things I thought about during the refactor below... it would be a good place to start testing. I've noted the ones I didn't test at all yet.

Cases to consider

  • optional params
  • minParams
  • function_named_arguments
  • nulls
  • implicit string casts
  • inline_casts
  • conflict fields
  • unsupported fields
  • dashboard params
  • licensing
  • in / not in

User facing changes

Checks removed

Several checks we used to do are gone now

  • function-specific validation checks
    • full-text search function validation
    • warnings for log functions and division by zero
  • making sure literals matched acceptedValues
  • making sure fields weren't in arguments where they shouldn't be (constantOnly)

Messages changed

Some error messages have been changed as well.

Wrong argument types

Most significantly, we no longer attempt to give per-parameter feedback. Instead, we list all the available signatures of the function, leaving the users to do the diffing themselves. This removes confusing messages like

Argument of [+] must be [date], found value [1] type [integer]

for

from a_index | eval 1 + "2".

It also largely follows the behavior of the TypeScript interpreter, though TypeScript goes further and I'm sure we could too.

Screenshot 2025-08-25 at 8 54 33 AM Screenshot 2025-08-25 at 8 54 41 AM
Optional param
Screenshot 2025-08-25 at 8 57 03 AM
Variable length params
Screenshot 2025-08-25 at 8 58 20 AM

Wrong arity

Screenshot 2025-08-06 at 1 03 51 PM Screenshot 2025-08-06 at 1 03 04 PM Screenshot 2025-08-06 at 1 02 20 PM

Validation ends if a child function generates errors

Screen.Recording.2025-08-01.at.2.04.54.PM.mov

Checklist

@drewdaemon drewdaemon requested a review from a team as a code owner August 25, 2025 17:51
@drewdaemon drewdaemon requested a review from stratoula August 25, 2025 22:55
Copy link
Contributor

@stratoula stratoula left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok everything looks fine except for the ts command bug. Let's fix it and I will do a final testing and approve! Thanx for addressing my feedback Drew!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah this is still broken

image

// used for testing... eventually should be moved to the __tests__ directory once
// kbn-esql-validation-autocomplete is merged into this package and this no longer
// has to be exported from this package
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@drewdaemon drewdaemon enabled auto-merge (squash) August 26, 2025 21:01
@elastic elastic deleted a comment from elasticmachine Aug 26, 2025
Copy link
Contributor

@stratoula stratoula left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last bug fixed! LGTM! Thanx for the effort Drew, I am very excited about this change!

(Btw at the end of our effort let's also check the issue with validation bugs, we might have fixed everything. In that case we can also close the issue. I am hoping that we will have less and less bugs and we can create individual issues for those)

* @param value - The array of string values to format.
* @returns The formatted list string.
*/
export function formatList(type: 'conjunction' | 'disjunction' | 'unit', value: string[]): string {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot find where this new method is used in this PR.

Are we planning to use it in a follow-up, or was this an oversight?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! I don't know why I couldn't find it! All good on my end.

Still, I'd like to rely on @Bamieh for the final approval

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me :elasticheart:

@afharo afharo requested a review from Bamieh August 27, 2025 09:38
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
onechat 438 440 +2
unifiedSearch 440 442 +2
total +4

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/esql-ast 517 516 -1
@kbn/i18n 27 30 +3
total +2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
esql 265.5KB 265.6KB +26.0B
onechat 788.7KB 788.3KB -441.0B
total -415.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
kbnUiSharedDeps-srcJs 3.8MB 3.8MB -3.3KB
Unknown metric groups

API count

id before after diff
@kbn/esql-ast 638 636 -2
@kbn/i18n 38 41 +3
total +1

References to deprecated APIs

id before after diff
@kbn/esql-validation-autocomplete 20 19 -1
Copy link
Contributor

@Bamieh Bamieh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i18n addition of formatList LGTM

@drewdaemon drewdaemon merged commit 1418b1c into elastic:main Aug 28, 2025
12 checks passed
qn895 pushed a commit to qn895/kibana that referenced this pull request Sep 2, 2025
## Summary

Closes elastic#180518

Part of elastic#221230

This is a full rewrite of our function validation logic. The primary
goal is to simplify our code, though the algorithm has also been
simplified and some user-facing validation messages have been changed.


## User facing changes

### Checks removed

Several checks we used to do are gone now
-  function-specific validation checks
   - full-text search function validation
   - warnings for log functions and division by zero
- making sure literals matched `acceptedValues`
- making sure fields weren't in arguments where they shouldn't be
(`constantOnly`)

### Messages changed

Some error messages have been changed as well.

#### Wrong argument types

Most significantly, we no longer attempt to give per-parameter feedback.
Instead, we list all the available signatures of the function, leaving
the users to do the diffing themselves. This removes confusing messages
like

`Argument of [+] must be [date], found value [1] type [integer]` 

for 

`from a_index | eval 1 + "2"`.

It also largely follows the behavior of the TypeScript interpreter,
though TypeScript goes further and I'm sure we could too.

<img width="657" height="370" alt="Screenshot 2025-08-25 at 8 54 33 AM"
src="https://github.com/user-attachments/assets/09c78fde-0c5b-444d-ad53-db62d3ac17b3"
/>
<img width="535" height="352" alt="Screenshot 2025-08-25 at 8 54 41 AM"
src="https://github.com/user-attachments/assets/b8750622-a0f4-482e-8da3-26efaacffdeb"
/>

##### Optional param

<img width="637" height="189" alt="Screenshot 2025-08-25 at 8 57 03 AM"
src="https://github.com/user-attachments/assets/ac3e5f8f-22e2-4829-b10c-48278e195074"
/>


##### Variable length params

<img width="632" height="286" alt="Screenshot 2025-08-25 at 8 58 20 AM"
src="https://github.com/user-attachments/assets/6f5fba8a-b3c9-42d3-a132-d77dc4cc642f"
/>

#### Wrong arity
<img width="566" height="125" alt="Screenshot 2025-08-06 at 1 03 51 PM"
src="https://github.com/user-attachments/assets/873a7867-fa61-415b-a7b1-f79790610a3a"
/>

<img width="635" height="58" alt="Screenshot 2025-08-06 at 1 03 04 PM"
src="https://github.com/user-attachments/assets/7386034d-a86f-4962-96e4-76bf50a793d1"
/>

<img width="729" height="136" alt="Screenshot 2025-08-06 at 1 02 20 PM"
src="https://github.com/user-attachments/assets/299f7430-7bc3-42f2-9ec7-7c969ef1515d"
/>

#### Validation ends if a child function generates errors


https://github.com/user-attachments/assets/5cb5a55d-63b7-4053-a606-ba769f46d516

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
drewdaemon added a commit that referenced this pull request Sep 11, 2025
## Summary

Reliably getting a list of call signatures that match a function's
arguments is a key operation used in both
- Detecting if the arguments are valid (validation)
- Detecting the return type of a function (expression type detection)

I noticed that we had vastly improved this logic for validation in
#230139, but `getExpressionType`
was still using a separate, inferior implementation, leading to bugs
related to incorrect expression type detection.

This PR extends the benefits of our improvements to expression type
detection.

### Example bug

#### Before

<img width="493" height="227" alt="Screenshot 2025-09-10 at 11 15 54 AM"
src="https://github.com/user-attachments/assets/69b4f544-be1f-4e8a-ba99-e3ace7d53e2b"
/>

_here `MV_SLICE` actually returns a keyword, but the expression
evaluator says it returns a `date`_

#### After
 
<img width="490" height="95" alt="Screenshot 2025-09-10 at 11 55 28 AM"
src="https://github.com/user-attachments/assets/1dac9a9e-a690-4e63-b166-3f468513cedc"
/>


### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
KodeRad pushed a commit to KodeRad/kibana that referenced this pull request Sep 15, 2025
## Summary

Reliably getting a list of call signatures that match a function's
arguments is a key operation used in both
- Detecting if the arguments are valid (validation)
- Detecting the return type of a function (expression type detection)

I noticed that we had vastly improved this logic for validation in
elastic#230139, but `getExpressionType`
was still using a separate, inferior implementation, leading to bugs
related to incorrect expression type detection.

This PR extends the benefits of our improvements to expression type
detection.

### Example bug

#### Before

<img width="493" height="227" alt="Screenshot 2025-09-10 at 11 15 54 AM"
src="https://github.com/user-attachments/assets/69b4f544-be1f-4e8a-ba99-e3ace7d53e2b"
/>

_here `MV_SLICE` actually returns a keyword, but the expression
evaluator says it returns a `date`_

#### After
 
<img width="490" height="95" alt="Screenshot 2025-09-10 at 11 55 28 AM"
src="https://github.com/user-attachments/assets/1dac9a9e-a690-4e63-b166-3f468513cedc"
/>


### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this pull request Sep 24, 2025
## Summary

Reliably getting a list of call signatures that match a function's
arguments is a key operation used in both
- Detecting if the arguments are valid (validation)
- Detecting the return type of a function (expression type detection)

I noticed that we had vastly improved this logic for validation in
elastic#230139, but `getExpressionType`
was still using a separate, inferior implementation, leading to bugs
related to incorrect expression type detection.

This PR extends the benefits of our improvements to expression type
detection.

### Example bug

#### Before

<img width="493" height="227" alt="Screenshot 2025-09-10 at 11 15 54 AM"
src="https://github.com/user-attachments/assets/69b4f544-be1f-4e8a-ba99-e3ace7d53e2b"
/>

_here `MV_SLICE` actually returns a keyword, but the expression
evaluator says it returns a `date`_

#### After
 
<img width="490" height="95" alt="Screenshot 2025-09-10 at 11 55 28 AM"
src="https://github.com/user-attachments/assets/1dac9a9e-a690-4e63-b166-3f468513cedc"
/>


### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
niros1 pushed a commit that referenced this pull request Sep 30, 2025
## Summary

Reliably getting a list of call signatures that match a function's
arguments is a key operation used in both
- Detecting if the arguments are valid (validation)
- Detecting the return type of a function (expression type detection)

I noticed that we had vastly improved this logic for validation in
#230139, but `getExpressionType`
was still using a separate, inferior implementation, leading to bugs
related to incorrect expression type detection.

This PR extends the benefits of our improvements to expression type
detection.

### Example bug

#### Before

<img width="493" height="227" alt="Screenshot 2025-09-10 at 11 15 54 AM"
src="https://github.com/user-attachments/assets/69b4f544-be1f-4e8a-ba99-e3ace7d53e2b"
/>

_here `MV_SLICE` actually returns a keyword, but the expression
evaluator says it returns a `date`_

#### After
 
<img width="490" height="95" alt="Screenshot 2025-09-10 at 11 55 28 AM"
src="https://github.com/user-attachments/assets/1dac9a9e-a690-4e63-b166-3f468513cedc"
/>


### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
@drewdaemon drewdaemon deleted the 221230/change-function-validation-scheme branch October 7, 2025 20:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting Feature:ES|QL ES|QL related features in Kibana release_note:enhancement Team:ESQL ES|QL related features in Kibana t// v9.2.0

7 participants