[ML] Adjust jinaai rerank response parser to handle document field as string or object by jonathan-buttner · Pull Request #136751 · elastic/elasticsearch

jonathan-buttner · 2025-10-17T14:14:00Z

This PR adds functionality to handle the document field as either an object or a string.

Originally we were only accepted the response format as:

{
     "id": "1983d114-a6e8-4940-b121-eb4ac3f6f703",
     "results": [
          {
               "document": {
                    "text": "Washington, D.C.  is the capital of the United States. It is a federal district."
               },
               "index": 2,
               "relevance_score": 0.98005307
          },
          {
               "document": {
                    "text": "abc."
               },
               "index": 3,
               "relevance_score": 0.27904198
          },
          {
               "document": {
                    "text": "Carson City is the capital city of the American state of Nevada."
               },
               "index": 0,
               "relevance_score": 0.10194652
          }
     ],
     "usage": {
          "total_tokens": 15
     }
}

Now we also support document as a string:

{
     "id": "1983d114-a6e8-4940-b121-eb4ac3f6f703",
     "results": [
          {
               "document": "Washington, D.C.  is the capital of the United States. It is a federal district.",
               "index": 2,
               "relevance_score": 0.98005307
          },
          {
               "document":  "abc",
               "index": 3,
               "relevance_score": 0.27904198
          },
          {
               "document": "Carson City is the capital city of the American state of Nevada.",
               "index": 0,
               "relevance_score": 0.10194652
          }
     ],
     "usage": {
          "total_tokens": 15
     }
}

elasticsearchmachine · 2025-10-17T14:15:30Z

Hi @jonathan-buttner, I've created a changelog YAML for you.

elasticsearchmachine · 2025-10-17T15:48:53Z

Pinging @elastic/ml-core (Team:ML)

DonalEvans

Just some optional formatting changes.

DonalEvans · 2025-10-17T16:00:29Z

...a/org/elasticsearch/xpack/inference/services/jinaai/response/JinaAIRerankResponseEntity.java

@@ -77,80 +74,78 @@ public class JinaAIRerankResponseEntity {
     * @throws IOException if there is an error parsing the response
     */
    public static InferenceServiceResults fromResponse(HttpResult response) throws IOException {


Would it be worth updating the javadoc for this method to include the new behaviour? If you do decide to update it, putting <pre> tags around the JSON parts would be helpful, to make the javadoc more readable in the IDE. Right now, when mousing over the method name to show the javadoc, the JSON is formatted all on one line, which is very difficult to read.

DonalEvans · 2025-10-17T16:12:43Z

...a/org/elasticsearch/xpack/inference/services/jinaai/response/JinaAIRerankResponseEntity.java

+    private record Response(List<ResultItem> results) {
+        @SuppressWarnings("unchecked")
+        public static final ConstructingObjectParser<Response, Void> PARSER = new ConstructingObjectParser<>(
+            Response.class.getSimpleName(),


Using getSimpleName() here will result in any errors during parsing referencing Response which is pretty vague if we need to use it for debugging. Using any of the other "name" methods would be too verbose, but is there some way we could include the name of the parent class? Or is that overkill?

Hmm, typically we just do the getSimpleName(). I think the error will have the stacktrace though which should get us to the code that is failing 🤔

DonalEvans · 2025-10-17T16:27:08Z

...a/org/elasticsearch/xpack/inference/services/jinaai/response/JinaAIRerankResponseEntity.java

-            } else {
-                parser.nextToken();
-            }
+    private record ResultItem(int index, float relevance_score, @Nullable Document document) {


Nitpick, but relevance_score should probably be relevanceScore for code style consistency.

DonalEvans · 2025-10-17T16:30:49Z

.../elasticsearch/xpack/inference/services/jinaai/response/JinaAIRerankResponseEntityTests.java

+    private final String WASHINGTON_TEXT = "Washington, D.C..";
+    private final String CAPITAL_PUNISHMENT_TEXT =
+        "Capital punishment has existed in the United States since before the United States was a country. ";
+    private final String CARSON_CITY_TEXT = "Carson City is the capital city of the American state of Nevada.";
+
+    private final List<RankedDocsResults.RankedDoc> responseLiteralDocsWithText = List.of(
+        new RankedDocsResults.RankedDoc(2, 0.98005307F, WASHINGTON_TEXT),
+        new RankedDocsResults.RankedDoc(3, 0.27904198F, CAPITAL_PUNISHMENT_TEXT),
+        new RankedDocsResults.RankedDoc(0, 0.10194652F, CARSON_CITY_TEXT)
+    );


Could these constants be moved to the top of the class and made static? Also, if you do make them all static, for style consistency, responseLiteralDocsWithText should be in all caps.

…i-rerank-fix

… string or object (elastic#136751) * Fixing rerank response parser * Update docs/changelog/136751.yaml * [CI] Auto commit changes from spotless * Addressing feedback * Updating comment about response format --------- Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>

elasticsearchmachine · 2025-10-17T18:11:51Z

💚 Backport successful

Status	Branch	Result
✅	9.2
✅	9.1

… string or object (elastic#136751) * Fixing rerank response parser * Update docs/changelog/136751.yaml * [CI] Auto commit changes from spotless * Addressing feedback * Updating comment about response format --------- Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>

… string or object (#136751) (#136762) * Fixing rerank response parser * Update docs/changelog/136751.yaml * [CI] Auto commit changes from spotless * Addressing feedback * Updating comment about response format --------- Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>

… string or object (#136751) (#136763) * Fixing rerank response parser * Update docs/changelog/136751.yaml * [CI] Auto commit changes from spotless * Addressing feedback * Updating comment about response format --------- Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>

… string or object (elastic#136751) * Fixing rerank response parser * Update docs/changelog/136751.yaml * [CI] Auto commit changes from spotless * Addressing feedback * Updating comment about response format --------- Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co> (cherry picked from commit a1d7b8a)

jonathan-buttner · 2025-10-17T19:16:13Z

💚 All backports created successfully

Status	Branch	Result
✅	8.19

Questions ?

Please refer to the Backport tool documentation

… string or object (#136751) (#136765) * Fixing rerank response parser * Update docs/changelog/136751.yaml * [CI] Auto commit changes from spotless * Addressing feedback * Updating comment about response format --------- (cherry picked from commit a1d7b8a) Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>

Fixing rerank response parser

08504a8

elasticsearchmachine added the v9.3.0 label Oct 17, 2025

jonathan-buttner added >bug :ml Machine learning Team:ML Meta label for the ML team labels Oct 17, 2025

Update docs/changelog/136751.yaml

53e3bb8

jonathan-buttner added v9.2.0 v8.19.6 v9.1.6 v9.2.1 auto-backport Automatically create backport pull requests when merged labels Oct 17, 2025

[CI] Auto commit changes from spotless

de81a70

jonathan-buttner marked this pull request as ready for review October 17, 2025 15:48

DonalEvans approved these changes Oct 17, 2025

View reviewed changes

jonathan-buttner added 3 commits October 17, 2025 12:56

Addressing feedback

c19114f

Merge branch 'main' of github.com:elastic/elasticsearch into ml-jinaa…

f5d2196

…i-rerank-fix

Updating comment about response format

91e981e

jonathan-buttner enabled auto-merge (squash) October 17, 2025 17:21

jonathan-buttner merged commit a1d7b8a into elastic:main Oct 17, 2025
34 checks passed

jonathan-buttner mentioned this pull request Oct 17, 2025

[9.2] [ML] Adjust jinaai rerank response parser to handle document field as string or object (#136751) #136762

Merged

jonathan-buttner mentioned this pull request Oct 17, 2025

[9.1] [ML] Adjust jinaai rerank response parser to handle document field as string or object (#136751) #136763

Merged

jonathan-buttner deleted the ml-jinaai-rerank-fix branch October 17, 2025 19:15

jonathan-buttner mentioned this pull request Oct 17, 2025

[8.19] [ML] Adjust jinaai rerank response parser to handle document field as string or object (#136751) #136765

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ML] Adjust jinaai rerank response parser to handle document field as string or object#136751

[ML] Adjust jinaai rerank response parser to handle document field as string or object#136751
jonathan-buttner merged 6 commits intoelastic:mainfrom
jonathan-buttner:ml-jinaai-rerank-fix

jonathan-buttner commented Oct 17, 2025

elasticsearchmachine commented Oct 17, 2025

elasticsearchmachine commented Oct 17, 2025

DonalEvans left a comment

DonalEvans Oct 17, 2025

DonalEvans Oct 17, 2025

jonathan-buttner Oct 17, 2025

DonalEvans Oct 17, 2025

DonalEvans Oct 17, 2025

Uh oh!

elasticsearchmachine commented Oct 17, 2025

jonathan-buttner commented Oct 17, 2025

Labels

3 participants

Conversation

jonathan-buttner commented Oct 17, 2025

elasticsearchmachine commented Oct 17, 2025

elasticsearchmachine commented Oct 17, 2025

DonalEvans left a comment

Choose a reason for hiding this comment

DonalEvans Oct 17, 2025

Choose a reason for hiding this comment

DonalEvans Oct 17, 2025

Choose a reason for hiding this comment

jonathan-buttner Oct 17, 2025

Choose a reason for hiding this comment

DonalEvans Oct 17, 2025

Choose a reason for hiding this comment

DonalEvans Oct 17, 2025

Choose a reason for hiding this comment

Uh oh!

elasticsearchmachine commented Oct 17, 2025

💚 Backport successful

jonathan-buttner commented Oct 17, 2025

💚 All backports created successfully

Questions ?

Labels

3 participants