feat: re-enable bfloat16 in semantic text#139347
Conversation
…omaios/elasticsearch into bfloat16_esql_and_semantic_text_tests
…_esql_and_semantic_text_tests
…_esql_and_semantic_text_tests
…_esql_and_semantic_text_tests
…omaios/elasticsearch into bfloat16_esql_and_semantic_text_tests
…omaios/elasticsearch into bfloat16_esql_and_semantic_text_tests
…omaios/elasticsearch into bfloat16_esql_and_semantic_text_tests
…omaios/elasticsearch into bfloat16_esql_and_semantic_text_tests
|
Hi @mromaios, I've created a changelog YAML for you. |
…ios/elasticsearch into enable_test_bfloat16_semantic_text
|
Pinging @elastic/search-relevance (Team:Search - Relevance) |
| MapperService bitMapperService = mapperServiceForFieldWithModelSettings( | ||
| fieldName, | ||
| inferenceId, | ||
| new MinimalServiceSettings( | ||
| "my-service", | ||
| TaskType.TEXT_EMBEDDING, | ||
| 1024, | ||
| SimilarityMeasure.L2_NORM, | ||
| DenseVectorFieldMapper.ElementType.BIT | ||
| ) | ||
| ); | ||
| assertMapperService.accept(bitMapperService, DenseVectorFieldMapper.ElementType.BIT); |
There was a problem hiding this comment.
We were missing this test case (in SemanticTextFieldMappterTests, at least), so I added it while I was already working in this area.
jimczi
left a comment
There was a problem hiding this comment.
I love the reactivity here but I don't think it's the right thing to do. We don't have models that return bfloat16 embeddings, they will all expose it as floats for now. The bfloat16 part is on the mapping side where models that return float are compatible with the element_type bfloat16. The idea is that models continue to generate floats but in the indexing we replace the floats with the bfloat16 transparently. So what we really need here is a way to change the element type in semantic text.
I'd also argue that we could make it the default for semantic text, maybe that should be the goal here, @benwtrent WDYT?
jimczi
left a comment
There was a problem hiding this comment.
We chatted offline and agreed that it's ok to handle the new element type at the inference endpoint level. That will be useful for future models that handle this format natively but we shouldn't oblige users to change their inference endpoint where only a mapping change is needed. Embedding models that output floats should continue to use the float element type and semantic text fields should expose a way to change that granularly (opting for bfloat16 at index time).
Co-authored-by: Mike Pellegrini <mike.pellegrini@elastic.co>
This PR re-enables bfloat16 element type configuration in
semantic_text, also adds a couple of tests for that