ESQL: Fix ReplaceMissingFieldsWithNull

[alex-spies]

Fix #126030
Fix #126036
Fix #121754

There are multiple problems with ReplaceMissingFieldsWithNull:

When encountering projections, it tries to do the job of field extraction for missing fields by injecting an Eval that creates a literal null with the same name id as the field attribute for the missing field. But:

1. We only insert an Eval in case that a Project relies on the missing attribute. There could be other plan nodes that rely on the missing attribute.
2. Even for Projects, we only insert an Eval in case we squarely project for the field - in case of aliases (e.g. from RENAME), we do nothing.
3. In case of multiple Projects that use this attribute, we create multiple attributes with the original field attribute's id, causing a wrong Layout. This triggered ES|QL: NullPointerException planning EVAL #121754. This could even happen if there's simply a command upstream from a single Project in case the upstream command triggers a field extraction for the missing field.

There are multiple ways of dealing with this:

1. simple solutions:
   1. We could move the creation of the Evals into InsertFieldExtraction. This will be correct and simple, but we will miss out on important logical optimizations, namely turning the EsRelation into a LocalRelation in case all fields are missing, or propagating the literal nulls created in place of the missing field (enabling more constant folding etc.).
   2. We could just place an Eval directly on top of any EsRelation that has missing fields, introducing literal nulls in place of the missing attributes early on. This has a risk of emitting null blocks unnecessarily early in the execution, but the logical optimizer should be able to deal with this.
2. more complex solutions:
   1. We could just avoid replacing the same field attribute twice. This does not address the problem that upstream query plan nodes might trigger a field extraction, leading to duplicate name ids again.
   2. We could keep the original logic but just use new attribute ids. This is what my first attempt (ESQL: Unique ids when replacing missing fields #125656) does; it seems to work fine, but downstream nodes need updating to use the Eval's newly created ReferenceAttributes instead of the original FieldAttributes, at least in terms of the name id of the ReferenceAttribute. But! We cannot just replace field attributes by reference attributes in downstream nodes, that can lead to class cast exceptions...
   3. Instead of injecting Evals just before Projects, we could inject them just before the first query plan node that needs them. This would replicate the logic in InsertFieldExtraction, but in the local logical optimizer. In addition to the duplicated logic, this has a risk of accidentally messing up the ordering of the output fields (Evals place their attributes always on the right hand side), which could cause serious problems unless there's a Project/Stats somewhere downstream (which is probably always given, but still).

This PR uses approach 1.ii. (Approach 1.i would also be possible and is arguably more natural. It would require double checking that we don't miss out on some optimizations, though.)

[elasticsearchmachine]

Hi @alex-spies, I've created a changelog YAML for you.

[fang-xing-esql]

Thank you @alex-spies, I added a couple of things that I can think of.

[alex-spies]

Thanks for the remarks @fang-xing-esql . I'll add a couple of comments to the code so that we don't have to scratch our heads with the same questions in the future.

[elasticsearchmachine]

Hi @alex-spies, I've updated the changelog YAML for you.

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[costin]

LGTM

[fang-xing-esql]

LGTM, thank you @alex-spies !

[alex-spies]

Last CI run was green except for elasticsearch-ci/part-1 timing out:

2025-04-02 20:08:37 CEST	> Task :build-tools:reaper:spotlessJava
2025-04-03 01:07:06 CEST	# Received cancellation signal, interrupting
2025-04-03 01:07:06 CEST	🚨 Error: The command was interrupted by a signal: signal: terminated

Restarted CI as there was a merge conflict now.

[alex-spies]

The checks here complain that they're still waiting on elasticsearch-ci/part-2, but in buildkite, this pipeline already passed green.

[alex-spies]

Seems like the reporting of the status of elasticsearch-ci/part-2 may be stuck.

But since the CI passed in CI, this is safe to merge.

[alex-spies]

Ah yeah, publishing the build scan failed:

BUILD SUCCESSFUL in 36m 35s
3025 actionable tasks: 1247 executed, 1652 from cache, 126 up-to-date
Publishing build scan...
Publishing build scan failed due to network error 'java.net.SocketTimeoutException: Read timed out' (2 retries remaining)...
Publishing build scan failed due to network error 'java.net.SocketTimeoutException: Read timed out' (1 retry remaining)...
A network error occurred.
If you require assistance with this problem, please report it to your Develocity administrator and include the following information via copy/paste.
----------
Gradle version: 8.13
Plugin version: 3.19.2
Request URL: https://gradle-enterprise.elastic.co/scans/publish/gradle/3.19.2/token
Request ID: dea4cbad-e35c-4b39-af24-a35bcbaf4292
Exception: java.net.SocketTimeoutException: Read timed out
----------