Implement support for weighted rrf

[mridula-s109]

Implement support for weighted RRF

Summary

This PR implements support for weighted RRF (Reciprocal Rank Fusion) retrievers, allowing users to specify custom weights for each sub-retriever within an RRF retriever configuration. This addresses a common customer request to customize the influence of different retrievers in the RRF scoring process.

Core Implementation

• Enhanced RRFRetrieverBuilder: Extended to support both weighted and non-weighted retriever configurations
• New RRFRetrieverComponent: Added weight validation and handling for individual retrievers
• Backward Compatibility: Maintains support for existing RRF configurations without weights (default weight = 1.0)

Key Features

1. Weighted Retriever Support

Users can now specify weights for individual retrievers:

{
  "retriever": {
    "rrf": {
      "retrievers": [
        {
          "retriever": {
            "standard": { 
              "query": { "match": { "description": "pizza" } } 
            }
          },
          "weight": 0.7
        },
        {
          "retriever": {
            "knn": { 
              "field": "vector", 
              "query_vector": [1,2,3], 
              "k": 10 
            }
          },
          "weight": 0.3
        }
      ]
    }
  }
}

[github-actions]

🔍 Preview links for changed docs

• docs/reference/query-languages/esql/kibana/docs/functions/score.md
• docs/internal/GeneralArchitectureGuide.md
• docs/reference/elasticsearch/configuration-reference/security-settings.md
• docs/reference/elasticsearch/index.md
• docs/reference/elasticsearch/mapping-reference/sparse-vector.md
• docs/reference/elasticsearch/rest-apis/index.md
• docs/reference/elasticsearch/rest-apis/retrievers.md
• docs/reference/query-languages/esql/limitations.md
• docs/reference/query-languages/query-dsl/query-dsl-bool-query.md
• docs/reference/query-languages/query-dsl/query-dsl-knn-query.md

More links …

• docs/reference/query-languages/query-dsl/query-dsl-sparse-vector-query.md
• docs/reference/query-languages/query-dsl/query-dsl-text-expansion-query.md
• docs/reference/query-languages/query-dsl/query-dsl-weighted-tokens-query.md
• docs/reference/search-connectors/elastic-managed-connectors.md
• docs/release-notes/breaking-changes.md
• docs/release-notes/deprecations.md
• docs/release-notes/index.md
• docs/release-notes/known-issues.md

[mridula-s109]

@ioanatia have addressed all the comments. please let me know if there are any more concerns.

[ioanatia]

this is getting super close! thank you for your patience 🙇‍♀️

[ioanatia]

nice work!

[Mikep86]

LGTM, nice work!

[flobernd]

Edit: nvm, the old syntax seems to be still valid.

@mridula-s109 @leemthompo Hey folks. Just double checking here. Judging from the provided JSON example, this seems to be a breaking change since the structure now includes a new nesting:

{
  "retriever": {
    "rrf": {
      "retrievers": [
        { // <- Retriever container
           "standard": { 
              "query": { "match": { "description": "pizza" } } 
            }
        },

to

{
  "retriever": {
    "rrf": {
      "retrievers": [
        { // <- New RRFRetrieverComponent class that wraps `retriever` and `weigth`
          "retriever": {
            "standard": { 
              "query": { "match": { "description": "pizza" } } 
            }
          },
          "weight": 0.7
        },

Is this expected to ship in a minor version?