ILM Explain: valid JSON on truncated step info by szybia · Pull Request #137638 · elastic/elasticsearch

szybia · 2025-11-05T16:58:35Z

Fix invalid JSON in _ilm/explain where previous_step_info is missing "} at the end, if it was truncated

Closes #135458

...plugin/core/src/test/java/org/elasticsearch/xpack/core/ilm/LifecycleExecutionStateTests.java

server/src/main/java/org/elasticsearch/cluster/metadata/LifecycleExecutionState.java

...plugin/core/src/test/java/org/elasticsearch/xpack/core/ilm/LifecycleExecutionStateTests.java

x-pack/plugin/ilm/src/main/java/org/elasticsearch/xpack/ilm/history/ILMHistoryItem.java

elasticsearchmachine · 2025-11-05T19:13:38Z

Hi @szybia, I've created a changelog YAML for you.

elasticsearchmachine · 2025-11-05T19:15:21Z

Pinging @elastic/es-data-management (Team:Data Management)

x-pack/plugin/ilm/src/main/java/org/elasticsearch/xpack/ilm/history/ILMHistoryItem.java

server/src/main/java/org/elasticsearch/cluster/metadata/LifecycleExecutionState.java

Copilot

Pull Request Overview

This PR fixes invalid JSON generation in the ILM _ilm/explain API when previous_step_info or step_info is truncated. Previously, truncated step info could result in malformed JSON missing closing "} characters. The fix ensures that truncated JSON strings are properly closed by appending "} after the truncation message.

Key changes:

Renamed truncateWithExplanation to potentiallyTruncateLongJsonWithExplanation with logic to always append "} when truncating
Updated truncation logic to avoid no-op cases where only the JSON ending would be truncated and then re-added
Removed redundant manual "} appending from ILMHistoryItem.failure()

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
server/src/main/java/org/elasticsearch/cluster/metadata/LifecycleExecutionState.java	Renamed truncation method and updated logic to append `"}` for valid JSON
x-pack/plugin/ilm/src/main/java/org/elasticsearch/xpack/ilm/history/ILMHistoryItem.java	Removed manual `"}` appending now handled by the new truncation method
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/ilm/LifecycleExecutionStateTests.java	Added test cases for the new truncation behavior and updated existing test expectations
x-pack/plugin/ilm/src/test/java/org/elasticsearch/xpack/ilm/action/TransportExplainLifecycleActionTests.java	Added integration test verifying JSON validity after truncation
docs/changelog/137638.yaml	Added changelog entry for the bug fix

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

...plugin/core/src/test/java/org/elasticsearch/xpack/core/ilm/LifecycleExecutionStateTests.java

…-json * upstream/main: Mute org.elasticsearch.xpack.inference.action.filter.ShardBulkInferenceActionFilterBasicLicenseIT testLicenseInvalidForInference {p0=false} elastic#137691 Mute org.elasticsearch.xpack.inference.action.filter.ShardBulkInferenceActionFilterBasicLicenseIT testLicenseInvalidForInference {p0=true} elastic#137690 [LTR] Fix feature display order when using explain. (elastic#137671) Remove extra RemoteClusterService instances in unit test (elastic#137647) Fix `ComponentTemplatesFileSettingsIT.testSettingsApplied` (elastic#137669) Consolidates troubleshooting content into the "Returning semantic field embeddings in _source" section (elastic#137233) Update bundled JDK to 25.0.1 (elastic#137640) resolve indices for prefixed _all expressions (elastic#137330) ESQL: Add TopN support for exponential histograms (elastic#137313) allows field caps to be cross project (elastic#137530) ESQL: Add exponential histogram percentile function (elastic#137553) Wait for nodes to have downloaded databases in `GeoIpDownloaderIT` (elastic#137636) Tighten on when THROTTLE decision can be returned (elastic#136794) Mute org.elasticsearch.xpack.esql.qa.single_node.GenerativeMetricsIT test elastic#137655 Add a test for two little known conditional processor paths (elastic#137645) Extract a common ORIGIN constant (elastic#137612) Remove early phase failure in batched (elastic#136889) Returning correct index mode from get data streams api (elastic#137646) [ML] Manage AD results indices (elastic#136065)

...m/src/test/java/org/elasticsearch/xpack/ilm/action/TransportExplainLifecycleActionTests.java

server/src/main/java/org/elasticsearch/cluster/metadata/LifecycleExecutionState.java

1. create better test 2. fix double-truncation discovered due to better test

…-json * upstream/main: (158 commits) Cleanup files from repo root folder (elastic#138030) Implement OpenShift AI integration for chat completion, embeddings, and reranking (elastic#136624) Optimize AsyncSearchErrorTraceIT to avoid failures (elastic#137716) Removes support for null TransportService in RemoteClusterService (elastic#137939) Mute org.elasticsearch.index.mapper.DateFieldMapperTests testSortShortcuts elastic#138018 rest-api-spec: fix type of enums (elastic#137521) Update Gradle wrapper to 9.2.0 (elastic#136155) Add RCS Strong Verification Documentation (elastic#137822) Use docvalue skippers on dimension fields (elastic#137029) Introduce INDEX_SHARD_COUNT_FORMAT (elastic#137210) Mute org.elasticsearch.xpack.inference.integration.AuthorizationTaskExecutorIT testCreatesChatCompletion_AndThenCreatesTextEmbedding elastic#138012 Fix ES|QL search context creation to use correct results type (elastic#137994) Improve Snapshot Logging (elastic#137470) Support extra output field in TOP function (elastic#135434) Remove NumericDoubleValues class (elastic#137884) [ML] Fix ML calendar event update scalability issues (elastic#136886) Task may be unregistered outside of the trace context in exceptional cases. (elastic#137865) Refine workaround for S3 repo analysis known issue (elastic#138000) Additional DEBUG logging on authc failures (elastic#137941) Cleanup index resolution (elastic#137867) ...

Copilot

Pull Request Overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.../ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/ExplainLifecycleIT.java

server/src/main/java/org/elasticsearch/cluster/metadata/LifecycleExecutionState.java

…-json * upstream/main: (247 commits) Mute org.elasticsearch.xpack.inference.integration.SemanticTextIndexOptionsIT testValidateIndexOptionsWithBasicLicense elastic#138513 Mute org.elasticsearch.xpack.esql.heap_attack.HeapAttackLookupJoinIT testLookupExplosionBigString elastic#138510 This shouldn't be zero (elastic#138501) sum of empty histogram is now null (elastic#138378) Test ES|QL bfloat16 support (elastic#138499) Fix exception handling in S3 `compareAndExchangeRegister` (elastic#138488) Mute org.elasticsearch.xpack.exponentialhistogram.ExponentialHistogramFieldMapperTests testFormattedDocValues elastic#138504 Mute org.elasticsearch.ingest.geoip.IngestGeoIpClientYamlTestSuiteIT test {yaml=ingest_geoip/60_ip_location_databases/Test adding, getting, and removing ip location databases} elastic#138502 ESQL: Refactor HeapAttackIT (elastic#138432) [Inference API] Add ElasticInferenceServiceDenseTextEmbeddingsServiceSettings to InferenceNamedWriteablesProvider (elastic#138484) Store split indices (elastic#138396) ES|QL Update CHUNK to support chunking_settings as optional argument (elastic#138123) Extract common blob-update logic in `S3HttpHandler` (elastic#138490) Cleanup esql request building api (elastic#138398) Round sum and avg in exponential_histogram CSV tests (elastic#138472) ESQL: load exponential_histogram total count as double instead of long (elastic#138417) [SIMD] Use fixed width native types for better Java interoperability (elastic#138429) Do not use Min or Max as Top's surrogate when there is an outputField (elastic#138380) ES|QL: Fix generative tests (elastic#138478) Mute org.elasticsearch.xpack.inference.integration.AuthorizationTaskExecutorIT testCreatesEisChatCompletion_DoesNotRemoveEndpointWhenNoLongerAuthorized elastic#138480 ...

...plugin/core/src/test/java/org/elasticsearch/xpack/core/ilm/LifecycleExecutionStateTests.java

.../ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/ExplainLifecycleIT.java

server/src/main/java/org/elasticsearch/cluster/metadata/LifecycleExecutionState.java

nielsbauman

LGTM! Thanks a lot for the work and the iterations! 🚀

elasticsearchmachine · 2025-11-25T16:10:56Z

💚 Backport successful

Status	Branch	Result
✅	9.2
✅	8.19
✅	9.1

ILM Explain: valid JSON on truncated step info

8ddeae5

elasticsearchmachine added the v9.3.0 label Nov 5, 2025

szybia commented Nov 5, 2025

View reviewed changes

szybia added >bug :Data Management/ILM+SLM DO NOT USE. Use ":StorageEngine/ILM" or ":Distributed Coordination/SLM" instead. labels Nov 5, 2025

Update docs/changelog/137638.yaml

1826916

szybia added auto-backport Automatically create backport pull requests when merged branch:9.2 branch:9.1 branch:8.19 labels Nov 5, 2025

elasticsearchmachine added v9.2.2 v8.19.8 v9.1.8 and removed branch:9.2 branch:9.1 branch:8.19 labels Nov 5, 2025

szybia marked this pull request as ready for review November 5, 2025 19:14

elasticsearchmachine added the Team:Data Management (obsolete) DO NOT USE. This team no longer exists. label Nov 5, 2025

prdoyle reviewed Nov 5, 2025

View reviewed changes

x-pack/plugin/ilm/src/main/java/org/elasticsearch/xpack/ilm/history/ILMHistoryItem.java Show resolved Hide resolved

nielsbauman reviewed Nov 6, 2025

View reviewed changes

server/src/main/java/org/elasticsearch/cluster/metadata/LifecycleExecutionState.java Outdated Show resolved Hide resolved

lukewhiting requested a review from Copilot November 6, 2025 12:40

Copilot AI reviewed Nov 6, 2025

View reviewed changes

...plugin/core/src/test/java/org/elasticsearch/xpack/core/ilm/LifecycleExecutionStateTests.java Outdated Show resolved Hide resolved

szybia added 3 commits November 6, 2025 15:03

simpler solution and improvements

7f4fcec

fix test

06dadee

szybia requested a review from nielsbauman November 6, 2025 17:46

nielsbauman reviewed Nov 10, 2025

View reviewed changes

niels comments

d2ff05b

1. create better test 2. fix double-truncation discovered due to better test

szybia requested a review from Copilot November 13, 2025 17:52

Copilot AI reviewed Nov 13, 2025

View reviewed changes

.../ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/ExplainLifecycleIT.java Outdated Show resolved Hide resolved

szybia commented Nov 13, 2025

View reviewed changes

server/src/main/java/org/elasticsearch/cluster/metadata/LifecycleExecutionState.java Outdated Show resolved Hide resolved

szybia requested a review from nielsbauman November 13, 2025 18:07

szybia added 3 commits November 13, 2025 18:33

change policy

8263ac9

niels suggestion

548784c

nielsbauman reviewed Nov 24, 2025

View reviewed changes

server/src/main/java/org/elasticsearch/cluster/metadata/LifecycleExecutionState.java Show resolved Hide resolved

niels niels

d86138b

szybia requested a review from nielsbauman November 24, 2025 19:07

nielsbauman approved these changes Nov 25, 2025

View reviewed changes

szybia merged commit 6763f4e into elastic:main Nov 25, 2025
35 checks passed

szybia deleted the ilm-explain-invalid-json branch November 25, 2025 16:10

This was referenced Nov 25, 2025

[9.2] ILM Explain: valid JSON on truncated step info (#137638) #138605

Merged

[8.19] ILM Explain: valid JSON on truncated step info (#137638) #138606

Merged

[9.1] ILM Explain: valid JSON on truncated step info (#137638) #138607

Merged

szybia added a commit to szybia/elasticsearch that referenced this pull request Nov 25, 2025

ILM Explain: valid JSON on truncated step info (elastic#137638)

7bf81b9

szybia added a commit to szybia/elasticsearch that referenced this pull request Nov 25, 2025

ILM Explain: valid JSON on truncated step info (elastic#137638)

8f38d08

szybia added a commit to szybia/elasticsearch that referenced this pull request Nov 25, 2025

ILM Explain: valid JSON on truncated step info (elastic#137638)

861d62c

elasticsearchmachine pushed a commit that referenced this pull request Nov 25, 2025

ILM Explain: valid JSON on truncated step info (#137638) (#138607)

55b2378

elasticsearchmachine pushed a commit that referenced this pull request Nov 25, 2025

ILM Explain: valid JSON on truncated step info (#137638) (#138606)

51a0709

ncordon pushed a commit to ncordon/elasticsearch that referenced this pull request Nov 26, 2025

ILM Explain: valid JSON on truncated step info (elastic#137638)

5d48289

elasticsearchmachine pushed a commit that referenced this pull request Nov 26, 2025

ILM Explain: valid JSON on truncated step info (#137638) (#138605)

6dfddc4

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ILM Explain: valid JSON on truncated step info#137638

ILM Explain: valid JSON on truncated step info#137638
szybia merged 11 commits intoelastic:mainfrom
szybia:ilm-explain-invalid-json

szybia commented Nov 5, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elasticsearchmachine commented Nov 5, 2025

elasticsearchmachine commented Nov 5, 2025

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nielsbauman left a comment

Uh oh!

elasticsearchmachine commented Nov 25, 2025

Labels

5 participants

Conversation

szybia commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elasticsearchmachine commented Nov 5, 2025

elasticsearchmachine commented Nov 5, 2025

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nielsbauman left a comment

Choose a reason for hiding this comment

Uh oh!

elasticsearchmachine commented Nov 25, 2025

💚 Backport successful

Labels

5 participants

szybia commented Nov 5, 2025 •

edited

Loading