Implement comprehensive top N parameter handling for text similarity reranker by Mikep86 · Pull Request #142039 · elastic/elasticsearch

Mikep86 · 2026-02-06T18:06:18Z

Currently, the text similarity reranker has, at best, spotty support for rerank services that can have a top_n (or similar) parameter set. This parameter configures the rerank service to return, at most, N reranked documents. When N is less than rank_window_size, it can lead to ArrayIndexOutOfBoundsExceptions in the current implementation. This is why we check that N is greater than or equal to rank_window_size, however the problem is that that the current check misses a bunch of rerank services that support top_n.

This PR addresses the problem by implementing comprehensive support for the top_n parameter:

Adds the TopNProvider interface, a unified way to report the value of the top_n parameter.
Implements unified and more robust logic for extracting scores from ranked docs. This logic also implements a friendlier failure mode when the number of ranked docs returned by the rerank service does not match what is expected, which can happen when a top_n parameter value is applied, but not properly reported.

TextSimilarityRankFeaturePhaseRankCoordinatorContext and its tests have also been simplified. With the unified score extraction logic, we no longer need to resolve chunk scoring configuration while computing scores, so that logic (and its related tests) have been removed.

…opNProvider

…ing to hide the top N setting.

…s have scores

…inatorContext

elasticsearchmachine · 2026-02-06T18:06:44Z

Hi @Mikep86, I've created a changelog YAML for you.

Mikep86 · 2026-02-06T19:19:18Z

...ugin/src/main/java/org/elasticsearch/xpack/inference/mock/TestRerankingServiceExtension.java

-                return new RankedDocsResults(results.stream().sorted(Comparator.reverseOrder()).toList());
+
+                // RankedDoc's compareTo implementation already sorts by score descending, so we don't need to reverse the sort order
+                var sortedResultsStream = results.stream().sorted();


So it turns out that our test reranker service has been sorting docs in reverse relevance this whole time 🫠 . We didn't catch it as an issue because ES sorts the results again (correctly) here. This all happens to work, as long as reranker reranks every doc sent to it. However, if the reranker truncates results (like if a top_n parameter is applied 😉 ), then the incorrect sort order becomes an issue.

Mikep86 · 2026-02-06T19:27:46Z

...pack/inference/rank/textsimilarity/TextSimilarityRankFeaturePhaseRankCoordinatorContext.java

+        // This method relies on callers filtering out feature docs with null feature data
+        assert Arrays.stream(featureDocs).noneMatch(featureDoc -> featureDoc.featureData == null);


We used to filter out feature docs with null featureData before generating the inference request, however there are two issues with that:

In all code paths, the caller already does this, so it's a redundant operation.

We don't consistently use the filtered feature doc array in computeScores. So, if a caller were to actually provide a featureDocs array with one or more docs with null featureData, we would still end up throwing an NPE somewhere in computeScores.

This assertion seems like a good balance between being explicit that callers must provide docs with non-null featureData and avoiding redundant operations.

Mikep86 · 2026-02-09T15:21:30Z

@elasticmachine update branch

kderusso

I have some concerns about backporting and immediately performing a followup but will defer to your team on that. Otherwise the changes look good to me, thanks for doing the cleanup!

kderusso · 2026-02-09T16:15:41Z

...in/java/org/elasticsearch/xpack/inference/rank/textsimilarity/TextSimilarityRankBuilder.java

-            chunkScorerConfig != null
-                ? new ChunkScorerConfig(chunkScorerConfig.size, inferenceText, chunkScorerConfig.chunkingSettings())
-                : null
+            failuresAllowed


Nice cleanup here!

kderusso · 2026-02-09T16:26:31Z

...pack/inference/rank/textsimilarity/TextSimilarityRankFeaturePhaseRankCoordinatorContext.java

+                    configuredTopN = topNProvider.getTopN();
                }
            }
            if (configuredTopN != null && configuredTopN < rankWindowSize) {


I like this suggestion. Talked about this a little with @Mikep86 offline, but I do have some concerns about backporting this change, and then immediately removing it for the better fix proposed here. That strategy will result in merge pain going forward for all future backports that touch this code.

I wonder if there's a way we could backport the change in a way that wouldn't cause a 5xx error in serverless but also not require the merge hell. I realize that manually enumerating top n supporting services isn't ideal but for the backport branches, now that we know the fix could we literally catch the OOB exception and return a friendlier message, then just go with this proposed fix going forward?

kderusso · 2026-02-09T16:27:41Z

...pack/inference/rank/textsimilarity/TextSimilarityRankFeaturePhaseRankCoordinatorContext.java

+        List<Integer> rankedDocToFeatureDoc = new ArrayList<>();
+        for (int i = 0; i < featureDocs.length; i++) {
+            RankFeatureDoc featureDoc = featureDocs[i];
+            for (int j = 0; j < featureDoc.featureData.size(); j++) {


Would it make sense to check anyway for safety?

kderusso

I have some concerns about backporting and immediately performing a followup but will defer to your team on that. Otherwise the changes look good to me, thanks for doing the cleanup!

davidkyle

LGTM

…ep86/elasticsearch into text-similarity-reranker_aioob-error

elasticsearchmachine · 2026-02-11T14:17:51Z

💔 Backport failed

Status	Branch	Result
❌	9.3	Commit could not be cherrypicked due to conflicts
❌	9.2	Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 142039

…reranker (elastic#142039) Currently, the text similarity reranker has, at best, spotty support for rerank services that can have a top_n (or similar) parameter set. This parameter configures the rerank service to return, at most, N reranked documents. When N is less than rank_window_size, it can lead to ArrayIndexOutOfBoundsExceptions in the current implementation. This is why we check that N is greater than or equal to rank_window_size, however the problem is that that the current check misses a bunch of rerank services that support top_n. This PR addresses the problem by implementing comprehensive support for the top_n parameter: - Adds the TopNProvider interface, a unified way to report the value of the top_n parameter. - Implements unified and more robust logic for extracting scores from ranked docs. This logic also implements a friendlier failure mode when the number of ranked docs returned by the rerank service does not match what is expected, which can happen when a top_n parameter value is applied, but not properly reported. TextSimilarityRankFeaturePhaseRankCoordinatorContext and its tests have also been simplified. With the unified score extraction logic, we no longer need to resolve chunk scoring configuration while computing scores, so that logic (and its related tests) have been removed. (cherry picked from commit 628c78d) # Conflicts: # x-pack/plugin/inference/qa/test-service-plugin/src/main/java/org/elasticsearch/xpack/inference/mock/TestRerankingServiceExtension.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/InferenceFeatures.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/rank/textsimilarity/TextSimilarityRankFeaturePhaseRankCoordinatorContext.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/rank/textsimilarity/TextSimilarityRankFeaturePhaseRankCoordinatorContextTests.java # x-pack/plugin/inference/src/yamlRestTest/resources/rest-api-spec/test/inference/70_text_similarity_rank_retriever.yml

Mikep86 · 2026-02-11T15:23:28Z

💚 All backports created successfully

Status	Branch	Result
✅	9.3
✅	9.2

Questions ?

Please refer to the Backport tool documentation

…reranker (elastic#142039) Currently, the text similarity reranker has, at best, spotty support for rerank services that can have a top_n (or similar) parameter set. This parameter configures the rerank service to return, at most, N reranked documents. When N is less than rank_window_size, it can lead to ArrayIndexOutOfBoundsExceptions in the current implementation. This is why we check that N is greater than or equal to rank_window_size, however the problem is that that the current check misses a bunch of rerank services that support top_n. This PR addresses the problem by implementing comprehensive support for the top_n parameter: - Adds the TopNProvider interface, a unified way to report the value of the top_n parameter. - Implements unified and more robust logic for extracting scores from ranked docs. This logic also implements a friendlier failure mode when the number of ranked docs returned by the rerank service does not match what is expected, which can happen when a top_n parameter value is applied, but not properly reported. TextSimilarityRankFeaturePhaseRankCoordinatorContext and its tests have also been simplified. With the unified score extraction logic, we no longer need to resolve chunk scoring configuration while computing scores, so that logic (and its related tests) have been removed. (cherry picked from commit 628c78d) # Conflicts: # x-pack/plugin/inference/qa/test-service-plugin/src/main/java/org/elasticsearch/xpack/inference/mock/TestRerankingServiceExtension.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/InferenceFeatures.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/rank/textsimilarity/TextSimilarityRankFeaturePhaseRankCoordinatorContext.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/openshiftai/rerank/OpenShiftAiRerankTaskSettings.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/rank/textsimilarity/TextSimilarityRankFeaturePhaseRankCoordinatorContextTests.java # x-pack/plugin/inference/src/yamlRestTest/resources/rest-api-spec/test/inference/70_text_similarity_rank_retriever.yml

…arity reranker (#142039) (#142314) * Implement comprehensive top N parameter handling for text similarity reranker (#142039) Currently, the text similarity reranker has, at best, spotty support for rerank services that can have a top_n (or similar) parameter set. This parameter configures the rerank service to return, at most, N reranked documents. When N is less than rank_window_size, it can lead to ArrayIndexOutOfBoundsExceptions in the current implementation. This is why we check that N is greater than or equal to rank_window_size, however the problem is that that the current check misses a bunch of rerank services that support top_n. This PR addresses the problem by implementing comprehensive support for the top_n parameter: - Adds the TopNProvider interface, a unified way to report the value of the top_n parameter. - Implements unified and more robust logic for extracting scores from ranked docs. This logic also implements a friendlier failure mode when the number of ranked docs returned by the rerank service does not match what is expected, which can happen when a top_n parameter value is applied, but not properly reported. TextSimilarityRankFeaturePhaseRankCoordinatorContext and its tests have also been simplified. With the unified score extraction logic, we no longer need to resolve chunk scoring configuration while computing scores, so that logic (and its related tests) have been removed. (cherry picked from commit 628c78d) # Conflicts: # x-pack/plugin/inference/qa/test-service-plugin/src/main/java/org/elasticsearch/xpack/inference/mock/TestRerankingServiceExtension.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/InferenceFeatures.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/rank/textsimilarity/TextSimilarityRankFeaturePhaseRankCoordinatorContext.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/rank/textsimilarity/TextSimilarityRankFeaturePhaseRankCoordinatorContextTests.java # x-pack/plugin/inference/src/yamlRestTest/resources/rest-api-spec/test/inference/70_text_similarity_rank_retriever.yml * [CI] Auto commit changes from spotless --------- Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>

…arity reranker (#142039) (#142317) * Implement comprehensive top N parameter handling for text similarity reranker (#142039) Currently, the text similarity reranker has, at best, spotty support for rerank services that can have a top_n (or similar) parameter set. This parameter configures the rerank service to return, at most, N reranked documents. When N is less than rank_window_size, it can lead to ArrayIndexOutOfBoundsExceptions in the current implementation. This is why we check that N is greater than or equal to rank_window_size, however the problem is that that the current check misses a bunch of rerank services that support top_n. This PR addresses the problem by implementing comprehensive support for the top_n parameter: - Adds the TopNProvider interface, a unified way to report the value of the top_n parameter. - Implements unified and more robust logic for extracting scores from ranked docs. This logic also implements a friendlier failure mode when the number of ranked docs returned by the rerank service does not match what is expected, which can happen when a top_n parameter value is applied, but not properly reported. TextSimilarityRankFeaturePhaseRankCoordinatorContext and its tests have also been simplified. With the unified score extraction logic, we no longer need to resolve chunk scoring configuration while computing scores, so that logic (and its related tests) have been removed. (cherry picked from commit 628c78d) # Conflicts: # x-pack/plugin/inference/qa/test-service-plugin/src/main/java/org/elasticsearch/xpack/inference/mock/TestRerankingServiceExtension.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/InferenceFeatures.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/rank/textsimilarity/TextSimilarityRankFeaturePhaseRankCoordinatorContext.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/openshiftai/rerank/OpenShiftAiRerankTaskSettings.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/rank/textsimilarity/TextSimilarityRankFeaturePhaseRankCoordinatorContextTests.java # x-pack/plugin/inference/src/yamlRestTest/resources/rest-api-spec/test/inference/70_text_similarity_rank_retriever.yml * [CI] Auto commit changes from spotless --------- Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>

…reranker (elastic#142039) Currently, the text similarity reranker has, at best, spotty support for rerank services that can have a top_n (or similar) parameter set. This parameter configures the rerank service to return, at most, N reranked documents. When N is less than rank_window_size, it can lead to ArrayIndexOutOfBoundsExceptions in the current implementation. This is why we check that N is greater than or equal to rank_window_size, however the problem is that that the current check misses a bunch of rerank services that support top_n. This PR addresses the problem by implementing comprehensive support for the top_n parameter: - Adds the TopNProvider interface, a unified way to report the value of the top_n parameter. - Implements unified and more robust logic for extracting scores from ranked docs. This logic also implements a friendlier failure mode when the number of ranked docs returned by the rerank service does not match what is expected, which can happen when a top_n parameter value is applied, but not properly reported. TextSimilarityRankFeaturePhaseRankCoordinatorContext and its tests have also been simplified. With the unified score extraction logic, we no longer need to resolve chunk scoring configuration while computing scores, so that logic (and its related tests) have been removed.

Mikep86 added 23 commits February 4, 2026 11:39

Added TODO

73cb2cf

Added top_n task setting to test reranking service

a61fe7b

Added YAML test

4422cab

Updated ranked doc score extraction

e3fef43

Don't reverse sort order

e75f55f

Added TopNProvider interface

fabafe5

Updated TextSimilarityRankFeaturePhaseRankCoordinatorContext to use T…

422caf5

…opNProvider

Spotless

8a97f42

Updated TestTaskSettings to implement TopNProvider. Also added a sett…

ab9023c

…ing to hide the top N setting.

Updated YAML tests

b01d8e4

Update extractScoresFromRankedDocs to detect when not all feature doc…

e62a551

…s have scores

Spotless

e3a7a09

Remove unnecessary validation check

5e2ca36

Don't resolve chunking settings

3944ed1

Remove ChunkScorerConfig from TextSimilarityRankFeaturePhaseRankCoord…

8bcdf0e

…inatorContext

Spotless

47bdb69

Fix unit tests

1a5e013

Added empty ranked docs test

202cffb

Assert on null feature data

ad28cab

Fix test

49e1b67

Added cluster feature

d9be5aa

Improved ranked doc size error check

de90b23

Merge branch 'main' into text-similarity-reranker_aioob-error

978062b

Mikep86 added >bug :Search Relevance/Ranking Scoring, rescoring, rank evaluation. v9.4.0 labels Feb 6, 2026

Update docs/changelog/142039.yaml

e724bc7

Mikep86 commented Feb 6, 2026

View reviewed changes

Add allow_rerank_failures test case

206959a

Mikep86 added auto-backport Automatically create backport pull requests when merged branch:9.2 branch:9.3 labels Feb 9, 2026

elasticsearchmachine added v9.3.1 v9.2.6 and removed branch:9.2 branch:9.3 labels Feb 9, 2026

Merge branch 'main' into text-similarity-reranker_aioob-error

5116dd9

kderusso approved these changes Feb 9, 2026

View reviewed changes

davidkyle approved these changes Feb 10, 2026

View reviewed changes

Mikep86 added 3 commits February 10, 2026 17:22

Gracefully handle a feature doc with no features

e62445f

Merge branch 'main' into text-similarity-reranker_aioob-error

1f661b7

Merge branch 'text-similarity-reranker_aioob-error' of github.com:Mik…

25b4367

…ep86/elasticsearch into text-similarity-reranker_aioob-error

Mikep86 merged commit 628c78d into elastic:main Feb 11, 2026
35 checks passed

elasticsearchmachine added the backport pending label Feb 11, 2026

Mikep86 mentioned this pull request Feb 11, 2026

[9.3] Implement comprehensive top N parameter handling for text similarity reranker (#142039) #142314

Merged

Mikep86 mentioned this pull request Feb 11, 2026

[9.2] Implement comprehensive top N parameter handling for text similarity reranker (#142039) #142317

Merged

Mikep86 removed the backport pending label Feb 11, 2026

Mikep86 mentioned this pull request Feb 11, 2026

[text_similarity_reranker]: Automatically extend top N when it is less than rank window size #142321

Open

DonalEvans mentioned this pull request Feb 25, 2026

rank-feature array index out of bounds exception #143007

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement comprehensive top N parameter handling for text similarity reranker#142039

Implement comprehensive top N parameter handling for text similarity reranker#142039
Mikep86 merged 29 commits intoelastic:mainfrom
Mikep86:text-similarity-reranker_aioob-error

Mikep86 commented Feb 6, 2026 •

edited

Loading

elasticsearchmachine commented Feb 6, 2026

Mikep86 Feb 6, 2026

Mikep86 Feb 6, 2026

Mikep86 commented Feb 9, 2026

kderusso left a comment

kderusso Feb 9, 2026

kderusso Feb 9, 2026

kderusso Feb 9, 2026

kderusso left a comment

davidkyle left a comment

Uh oh!

elasticsearchmachine commented Feb 11, 2026

Mikep86 commented Feb 11, 2026

Labels

8 participants

		// This method relies on callers filtering out feature docs with null feature data
		assert Arrays.stream(featureDocs).noneMatch(featureDoc -> featureDoc.featureData == null);

Conversation

Mikep86 commented Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

elasticsearchmachine commented Feb 6, 2026

Mikep86 Feb 6, 2026

Choose a reason for hiding this comment

Mikep86 Feb 6, 2026

Choose a reason for hiding this comment

Mikep86 commented Feb 9, 2026

kderusso left a comment

Choose a reason for hiding this comment

kderusso Feb 9, 2026

Choose a reason for hiding this comment

kderusso Feb 9, 2026

Choose a reason for hiding this comment

kderusso Feb 9, 2026

Choose a reason for hiding this comment

kderusso left a comment

Choose a reason for hiding this comment

davidkyle left a comment

Choose a reason for hiding this comment

Uh oh!

elasticsearchmachine commented Feb 11, 2026

💔 Backport failed

Mikep86 commented Feb 11, 2026

💚 All backports created successfully

Questions ?

Labels

8 participants

Mikep86 commented Feb 6, 2026 •

edited

Loading