Update sparse_vector field mapping to include default setting for token pruning

[markjhoy]

Updates the SparseVectorFieldMapper type to include index options for pruning tokens and associated configuration values.

Before this update, token pruning for sparse vector types is only available via the query (see parameters for the sparse vector query ).

With this PR, by default, any new indices with a sparse_vector field type will by default have token pruning turned on (previous indices that may have had sparse_vector fields that exist before this update will still keep pruning turned off by default). Any sparse_vector queries that have explicit pruning options will still override the index defaults if they are set up.

Example:

{
  "properties": {
    "example_field": {
       "type": "sparse_vector",
        "index_options": {
          "prune": (boolean, default is `true`),
          "pruning_config": {
            "tokens_freq_ratio_threshold": (integer, range 1-100, default is 5),
            "tokens_weight_threshold": (double, range 0.0-1.0, default if 0.4)
          }
        }
     }
  }
}

[elasticsearchmachine]

Hi @markjhoy, I've created a changelog YAML for you.

[markjhoy]

Note - due to the scope of the changes (and especially for the transport and index versions) - this will require a manual backport to 8.19

[elasticsearchmachine]

Pinging @elastic/search-eng (Team:SearchOrg)

[elasticsearchmachine]

Pinging @elastic/search-relevance (Team:Search - Relevance)

[elasticsearchmachine]

Pinging @elastic/es-search-foundations (Team:Search Foundations)

[markjhoy]

buildkite test this

[elasticsearchmachine]

💔 Backport failed

Status	Branch	Result
❌	8.19	Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 129089

[markjhoy]

💚 All backports created successfully

Status	Branch	Result
✅	8.19

Questions ?

Please refer to the Backport tool documentation