Skip to content

[ES|QL] Append the casting only when necessary in Discover filtering #234748

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
stratoula merged 4 commits into from
Sep 15, 2025
Merged

[ES|QL] Append the casting only when necessary in Discover filtering #234748

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4d37f89
[ES|QL] Append the casting only when necessary
stratoula Sep 11, 2025
ec1aa45
Merge branch 'main' into update-casting
stratoula Sep 15, 2025
609b909
Save in different function
stratoula Sep 15, 2025
12ecbed
Simplifications
stratoula Sep 15, 2025
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/platform/packages/shared/kbn-esql-ast/src/definitions/utils/expressions.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,8 @@ export function getExpressionType(

// #region signature matching

// THESE types are types that now do not need implicit casting to string
// ToDo: Check where it is being used and fix the usage to remove the need for this
export const PARAM_TYPES_THAT_SUPPORT_IMPLICIT_STRING_CASTING: FunctionParameterType[] = [
'date',
'date_nanos',
Expand Down
6 changes: 3 additions & 3 deletions src/platform/packages/shared/kbn-esql-utils/src/utils/append_to_query.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,10 +51,10 @@ describe('appendToQuery', () => {

it('appends a where clause in an existing query with casting to string when the type is not string or number', () => {
expect(
appendWhereClauseToESQLQuery('from logstash-* // meow', 'dest', 'tada!', '-', 'ip')
appendWhereClauseToESQLQuery('from logstash-* // meow', 'ip_field', 'tada!', '-', 'ip')
).toBe(
`from logstash-* // meow
| WHERE \`dest\`::string!="tada!"`
| WHERE \`ip_field\`!="tada!"`
);
});

Expand Down Expand Up @@ -169,7 +169,7 @@ AND \`dest\`=="Crete"`
)
).toBe(
`from logstash-* | where CIDR_MATCH(ip1, "127.0.0.2/32", "127.0.0.3/32")
and \`ip\`::string!="127.0.0.2/32"`
and \`ip\`!="127.0.0.2/32"`
);
});

Expand Down
15 changes: 12 additions & 3 deletions src/platform/packages/shared/kbn-esql-utils/src/utils/append_to_query.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,16 @@
import { parse, mutate, BasicPrettyPrinter } from '@kbn/esql-ast';
import { sanitazeESQLInput } from './sanitaze_input';

const PARAM_TYPES_NO_NEED_IMPLICIT_STRING_CASTING = [
'date',
'date_nanos',
'version',
'ip',
'boolean',
'number',
'string',
];

// Append in a new line the appended text to take care of the case where the user adds a comment at the end of the query
// in these cases a base query such as "from index // comment" will result in errors or wrong data if we don't append in a new line
export function appendToESQLQuery(baseESQLQuery: string, appendedText: string): string {
Expand Down Expand Up @@ -46,10 +56,9 @@ export function appendWhereClauseToESQLQuery(
// Adding the backticks here are they are needed for special char fields
let fieldName = sanitazeESQLInput(field);

// casting to string
// there are some field types such as the ip that need
// Casting to string: There are some field types that need
// to cast in string first otherwise ES will fail
if (fieldType !== 'string' && fieldType !== 'number' && fieldType !== 'boolean') {
if (fieldType === undefined || !PARAM_TYPES_NO_NEED_IMPLICIT_STRING_CASTING.includes(fieldType)) {
fieldName = `${fieldName}::string`;
}

Expand Down