[ES|QL] improve function validation#230139
Conversation
stratoula
left a comment
There was a problem hiding this comment.
ok everything looks fine except for the ts command bug. Let's fix it and I will do a final testing and approve! Thanx for addressing my feedback Drew!
...form/packages/shared/kbn-esql-ast/src/commands_registry/commands/completion/validate.test.ts
Show resolved
Hide resolved
src/platform/packages/shared/kbn-esql-ast/src/commands_registry/commands/eval/validate.test.ts
Show resolved
Hide resolved
|
|
||
| // used for testing... eventually should be moved to the __tests__ directory once | ||
| // kbn-esql-validation-autocomplete is merged into this package and this no longer | ||
| // has to be exported from this package |
stratoula
left a comment
There was a problem hiding this comment.
Last bug fixed! LGTM! Thanx for the effort Drew, I am very excited about this change!
(Btw at the end of our effort let's also check the issue with validation bugs, we might have fixed everything. In that case we can also close the issue. I am hoping that we will have less and less bugs and we can create individual issues for those)
| * @param value - The array of string values to format. | ||
| * @returns The formatted list string. | ||
| */ | ||
| export function formatList(type: 'conjunction' | 'disjunction' | 'unit', value: string[]): string { |
There was a problem hiding this comment.
I cannot find where this new method is used in this PR.
Are we planning to use it in a follow-up, or was this an oversight?
There was a problem hiding this comment.
There was a problem hiding this comment.
Thank you! I don't know why I couldn't find it! All good on my end.
Still, I'd like to rely on @Bamieh for the final approval
There was a problem hiding this comment.
Looks good to me :elasticheart:
💚 Build Succeeded
Metrics [docs]Module Count
Public APIs missing comments
Async chunks
Page load bundle
|
Bamieh
left a comment
There was a problem hiding this comment.
i18n addition of formatList LGTM
## Summary Closes elastic#180518 Part of elastic#221230 This is a full rewrite of our function validation logic. The primary goal is to simplify our code, though the algorithm has also been simplified and some user-facing validation messages have been changed. ## User facing changes ### Checks removed Several checks we used to do are gone now - function-specific validation checks - full-text search function validation - warnings for log functions and division by zero - making sure literals matched `acceptedValues` - making sure fields weren't in arguments where they shouldn't be (`constantOnly`) ### Messages changed Some error messages have been changed as well. #### Wrong argument types Most significantly, we no longer attempt to give per-parameter feedback. Instead, we list all the available signatures of the function, leaving the users to do the diffing themselves. This removes confusing messages like `Argument of [+] must be [date], found value [1] type [integer]` for `from a_index | eval 1 + "2"`. It also largely follows the behavior of the TypeScript interpreter, though TypeScript goes further and I'm sure we could too. <img width="657" height="370" alt="Screenshot 2025-08-25 at 8 54 33 AM" src="https://github.com/user-attachments/assets/09c78fde-0c5b-444d-ad53-db62d3ac17b3" /> <img width="535" height="352" alt="Screenshot 2025-08-25 at 8 54 41 AM" src="https://github.com/user-attachments/assets/b8750622-a0f4-482e-8da3-26efaacffdeb" /> ##### Optional param <img width="637" height="189" alt="Screenshot 2025-08-25 at 8 57 03 AM" src="https://github.com/user-attachments/assets/ac3e5f8f-22e2-4829-b10c-48278e195074" /> ##### Variable length params <img width="632" height="286" alt="Screenshot 2025-08-25 at 8 58 20 AM" src="https://github.com/user-attachments/assets/6f5fba8a-b3c9-42d3-a132-d77dc4cc642f" /> #### Wrong arity <img width="566" height="125" alt="Screenshot 2025-08-06 at 1 03 51 PM" src="https://github.com/user-attachments/assets/873a7867-fa61-415b-a7b1-f79790610a3a" /> <img width="635" height="58" alt="Screenshot 2025-08-06 at 1 03 04 PM" src="https://github.com/user-attachments/assets/7386034d-a86f-4962-96e4-76bf50a793d1" /> <img width="729" height="136" alt="Screenshot 2025-08-06 at 1 02 20 PM" src="https://github.com/user-attachments/assets/299f7430-7bc3-42f2-9ec7-7c969ef1515d" /> #### Validation ends if a child function generates errors https://github.com/user-attachments/assets/5cb5a55d-63b7-4053-a606-ba769f46d516 ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
## Summary Reliably getting a list of call signatures that match a function's arguments is a key operation used in both - Detecting if the arguments are valid (validation) - Detecting the return type of a function (expression type detection) I noticed that we had vastly improved this logic for validation in #230139, but `getExpressionType` was still using a separate, inferior implementation, leading to bugs related to incorrect expression type detection. This PR extends the benefits of our improvements to expression type detection. ### Example bug #### Before <img width="493" height="227" alt="Screenshot 2025-09-10 at 11 15 54 AM" src="https://github.com/user-attachments/assets/69b4f544-be1f-4e8a-ba99-e3ace7d53e2b" /> _here `MV_SLICE` actually returns a keyword, but the expression evaluator says it returns a `date`_ #### After <img width="490" height="95" alt="Screenshot 2025-09-10 at 11 55 28 AM" src="https://github.com/user-attachments/assets/1dac9a9e-a690-4e63-b166-3f468513cedc" /> ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Summary Reliably getting a list of call signatures that match a function's arguments is a key operation used in both - Detecting if the arguments are valid (validation) - Detecting the return type of a function (expression type detection) I noticed that we had vastly improved this logic for validation in elastic#230139, but `getExpressionType` was still using a separate, inferior implementation, leading to bugs related to incorrect expression type detection. This PR extends the benefits of our improvements to expression type detection. ### Example bug #### Before <img width="493" height="227" alt="Screenshot 2025-09-10 at 11 15 54 AM" src="https://github.com/user-attachments/assets/69b4f544-be1f-4e8a-ba99-e3ace7d53e2b" /> _here `MV_SLICE` actually returns a keyword, but the expression evaluator says it returns a `date`_ #### After <img width="490" height="95" alt="Screenshot 2025-09-10 at 11 55 28 AM" src="https://github.com/user-attachments/assets/1dac9a9e-a690-4e63-b166-3f468513cedc" /> ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Summary Reliably getting a list of call signatures that match a function's arguments is a key operation used in both - Detecting if the arguments are valid (validation) - Detecting the return type of a function (expression type detection) I noticed that we had vastly improved this logic for validation in elastic#230139, but `getExpressionType` was still using a separate, inferior implementation, leading to bugs related to incorrect expression type detection. This PR extends the benefits of our improvements to expression type detection. ### Example bug #### Before <img width="493" height="227" alt="Screenshot 2025-09-10 at 11 15 54 AM" src="https://github.com/user-attachments/assets/69b4f544-be1f-4e8a-ba99-e3ace7d53e2b" /> _here `MV_SLICE` actually returns a keyword, but the expression evaluator says it returns a `date`_ #### After <img width="490" height="95" alt="Screenshot 2025-09-10 at 11 55 28 AM" src="https://github.com/user-attachments/assets/1dac9a9e-a690-4e63-b166-3f468513cedc" /> ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Summary Reliably getting a list of call signatures that match a function's arguments is a key operation used in both - Detecting if the arguments are valid (validation) - Detecting the return type of a function (expression type detection) I noticed that we had vastly improved this logic for validation in #230139, but `getExpressionType` was still using a separate, inferior implementation, leading to bugs related to incorrect expression type detection. This PR extends the benefits of our improvements to expression type detection. ### Example bug #### Before <img width="493" height="227" alt="Screenshot 2025-09-10 at 11 15 54 AM" src="https://github.com/user-attachments/assets/69b4f544-be1f-4e8a-ba99-e3ace7d53e2b" /> _here `MV_SLICE` actually returns a keyword, but the expression evaluator says it returns a `date`_ #### After <img width="490" height="95" alt="Screenshot 2025-09-10 at 11 55 28 AM" src="https://github.com/user-attachments/assets/1dac9a9e-a690-4e63-b166-3f468513cedc" /> ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios

Summary
Closes #180518
Part of #221230
This is a full rewrite of our function validation logic. The primary goal is to simplify our code, though the algorithm has also been simplified and some user-facing validation messages have been changed.
For reviewers
I know... it looks big 😞 But most of it is updating tests! ✨
The place to start is src/platform/packages/shared/kbn-esql-ast/src/definitions/utils/validation/function.ts. All the other changes stem from this rewrite.
I did come across test suites that weren't worth updating and I removed those. I've tried to leave comments where I felt that was the case. I did significantly extend our test coverage in the
functions.test.tsfile where most of our function tests should live.I tried to be careful, but I wouldn't be surprised if there are some regressions. I have included a list of things I thought about during the refactor below... it would be a good place to start testing. I've noted the ones I didn't test at all yet.
Cases to consider
User facing changes
Checks removed
Several checks we used to do are gone now
acceptedValuesconstantOnly)Messages changed
Some error messages have been changed as well.
Wrong argument types
Most significantly, we no longer attempt to give per-parameter feedback. Instead, we list all the available signatures of the function, leaving the users to do the diffing themselves. This removes confusing messages like
Argument of [+] must be [date], found value [1] type [integer]for
from a_index | eval 1 + "2".It also largely follows the behavior of the TypeScript interpreter, though TypeScript goes further and I'm sure we could too.
Optional param
Variable length params
Wrong arity
Validation ends if a child function generates errors
Screen.Recording.2025-08-01.at.2.04.54.PM.mov
Checklist