Updates to allow using Cohere binary embedding response in semantic search queries.

[ymao1]

Summary

Updates to get semantic search working with bit embeddings.

Verify

Following the instructions in this tutorial:

This will verify that the process of using a inference endpoint with binary embedding works with semantic text queries.

1. Create a Cohere inference endpoint using bit embedding types

PUT _inference/text_embedding/cohere_bit
{
    "service": "cohere",
    "service_settings": {
        "api_key": <apiKey>,
        "model_id": "embed-english-v3.0",
        "embedding_type": "bit"
    }
}

2. Create an index with a semantic text mapping using the inference ID

PUT semantic-embeddings-bit
{
  "mappings": {
    "properties": {
      "content": { 
        "type": "semantic_text",
        "inference_id": "cohere_bit"
      }
    }
  }
}

3. Load data as directed by the the tutorial

4. Reindex the data

POST _reindex?wait_for_completion=false
{
  "source": {
    "index": "test-data",
    "size": 100 
  },
  "dest": {
    "index": "semantic-embeddings-bit"
  }
}

5. Run a query

GET semantic-embeddings-bit/_search
{
  "query": {
    "semantic": {
      "field": "content", 
      "query": "How to avoid muscle soreness while running?" 
    }
  }
}

[davidkyle]

LGTM

[elasticsearchmachine]

Hi @ymao1, I've created a changelog YAML for you.

[ymao1]

Waiting for @davidkyle to give a 👍 after running some validation tests.

[Mikep86]

Let's make sure to update this error message with the correct dimension count as well

[elasticsearchmachine]

Pinging @elastic/ml-core (Team:ML)

[elasticsearchmachine]

💔 Backport failed

Status	Branch	Result
❌	8.x	Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 121827

[ymao1]

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Questions ?

Please refer to the Backport tool documentation

[ymao1]

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Questions ?

Please refer to the Backport tool documentation