EQL: backport fix for async missing events and re-enable the feature

[luigidellaquila]

This PR backports #97718 to v 8.9.1

The problem: missing events were represented as null values in intermediate states (before converting them to {missing: true} JSON), but in some circumstances events have to be serialized, eg. when executing an async search, and that caused a NPE.

The fix: this PR replaces the null placeholder with a constant Event instance to represent missing events, allowing proper serialization.

Backport details: In the main branch, the fix also includes a TransportVersion change to add the new "missing" attribute to the serialization. Since backporting this part is particularly tricky, here we are using a work-around to avoid a version increment: we'll consider events with _index: "" (empty index) as missing events; the missing attribute won't be serialized, preserving the existing serialization format.

[elasticsearchmachine]

Pinging @elastic/es-ql (Team:QL)

[elasticsearchmachine]

Hi @luigidellaquila, I've created a changelog YAML for you.

[ChrisHegarty]

..
Backport details: In the main branch, the fix also includes a TransportVersion change to add the new "missing" attribute to the serialization. Since backporting this part is particularly tricky, here we are using a work-around to avoid a version increment: we'll consider events with _index: "" (empty index) as missing events; the missing attribute won't be serialized, preserving the existing serialization format.

In this particular case, I agree that the workaround ( as you have described ) is the safer way to proceed, rather than trying to determine the Transport range to eliminate and individual checks to update in main.

[astefan]

LGTM

[luigidellaquila]

Thank you @ChrisHegarty @astefan!
Merging