ESQL: Performance improvements for Lookup Join on Expression

[julian-elastic]

Implement two performance improvements for Lookup Join on Expression

1. Do not send duplicate columns more than once to the lookup node. Example column left_id will be just sent once instead of 3 times, even though it is used 3 times in the expression
2. Use the termsQueryList that we build for field based join for Expression Based Joins too, if it uses equals operator. This allows us to take advantage of the optimizations made for field based join and apply them for expression based join whenever possible. We end up doing less work per row, so we expect this to be faster.

Covered by existing UTs lookup-join-expression.LookupJoinOnSameLeftFieldTwice and multiple UTs on equals.

Benchmarks show only moderate improvement 50sec to 43sec (14% improvement) in esql_lookup_join_expression_eq_100k_keys_where_few_matching_like_limit_10000. See #135036 (comment)

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[elasticsearchmachine]

Hi @julian-elastic, I've created a changelog YAML for you.

[julian-elastic]

Buildkite benchmark this with esql-joins please

[julian-elastic]

Buildkite benchmark this with esql-joins please

[julian-elastic]

Buildkite benchmark this with esql-joins please

[julian-elastic]

Buildkite benchmark this with esql-joins please

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: e037cbb
• Baseline: d1f8bba (env ID 6ef55f83-5fc7-4df5-89ef-d646af159b9c)
• Contender: e037cbb (env ID 0fd3d972-b9e3-4e34-9c9f-ec46e7c41363)
• Benchmark results

This build ran two esql-joins benchmarks to evaluate performance impact of this PR.

History

• 💔 Build #20 failed 20c7fa5
• 💔 Build #19 failed 09c1de2
• 💔 Build #18 failed 09c1de2

cc @julian-elastic