New rescorer based on script

[limingnihao]

Introduce a new rescorer based on script.
A script can use scores from the previous query/rescorer.

For example:

GET books_index /_search
{
  "query": {
    "match": {
      "title": {
        "query": "ethics of ambiguity"
      }
    }
  },
  "rescore": {
    "window_size": 5,
    "script": {
      "script": {
        "source": "_score + (doc['num_likes'].value + doc['num_reviews'].value) * params.multiplier",
        "params": {
          "multiplier": 12
        }
      }
    }
  }
}

Closes #52338

[cla-checker-service]

💚 CLA has been signed

[limingnihao]

/test

[elasticmachine]

Pinging @elastic/es-search (Team:Search)

[amylashley]

Curious if there is any plan to merge this in the near future? It would be incredibly helpful for my work!

[jdconrad]

@javanna Any thoughts on possibly reviving this pr?

[javanna]

I am +1 on reviving this, we just did not get to it until now, sadly.

[davefernig]

I have two questions about this PR:

1. What is the expected behaviour in the case in which the argument to rescore is an array? Will _score refer to the score from the previous stage – whatever that happens to be? (In my case that would be great). The concrete example I'm interested in consists in two rescore stages:

• The first of which implements the sltr query of the elasticsearch-learning-to-rank plugin
• The second implements a sigmoid transform

{
  "size": 50,
  "query": {"match": {"title": {"query": "raymond chandler"}}},
  "rescore": [
    {
      "window_size": 500,
      "query": {
        "rescore_query": {
          "sltr": {
            "params": {
              "query_string": "raymond chandler"
            },
            "model": "lambdamart_v001",
            "cache": true
          }
        },
        "rescore_query_weight": 1,
        "query_weight": 0
      }
    },
    {
      "window_size": 50,
      "script": {
        "lang": "painless",
        "source": "source": "1.0 / (1.0 + Math.exp(0-_score))"
      }
    }
  ]
}

My hope is that the resulting scores at the end of this will be whatever SLTR returns, pushed through a sigmoid. (This can't be done in a single step, for the reason discussed here: o19s/elasticsearch-learning-to-rank#121 (comment) )

2. If the answer to 1 is "yes, this commit will solve all your problems," do we have a sense of if/when this will get merged?

cc @javanna @jdconrad

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)

[mayya-sharipova]

@elasticmachine generate changelog

[john-wagster]

question for my knowledge. I'm only somewhat familiar with Transport Versions. What does this protect? Does this mean we can only support this syntax in the next release forward? And why is that in this case as in is it possible to have some new Builder where the minimal supported version is any version?

[mayya-sharipova]

Good question. This ensures that node that de-serializes this request has the provided min version. If it doesn't, the request with a script rescorer fails (I think with IllegalArgumentException).
We could potentially add some logic, if the deserialization happens on an older node, convert it to query rescorer, but I think it makes the code more complicated for little added advantage: supporting mixed cluster scenarios.

[john-wagster]

makes sense thanks!

[john-wagster]

lgtm

[mayya-sharipova]

@elasticmachine test this please

[mayya-sharipova]

@elasticmachine test this please

[mayya-sharipova]

@elasticmachine test this please

[mayya-sharipova]

@elasticmachine test this please

[mayya-sharipova]

@elasticmachine test this please