Skip to content

feat: re-enable bfloat16 in semantic text#139347

Merged
Mikep86 merged 38 commits intoelastic:mainfrom
mromaios:enable_test_bfloat16_semantic_text
Dec 12, 2025
Merged

feat: re-enable bfloat16 in semantic text#139347
Mikep86 merged 38 commits intoelastic:mainfrom
mromaios:enable_test_bfloat16_semantic_text

Conversation

@mromaios
Copy link
Contributor

@mromaios mromaios commented Dec 11, 2025

This PR re-enables bfloat16 element type configuration in semantic_text, also adds a couple of tests for that

mromaios and others added 29 commits December 5, 2025 13:07
…omaios/elasticsearch into bfloat16_esql_and_semantic_text_tests
…omaios/elasticsearch into bfloat16_esql_and_semantic_text_tests
…omaios/elasticsearch into bfloat16_esql_and_semantic_text_tests
…omaios/elasticsearch into bfloat16_esql_and_semantic_text_tests
…omaios/elasticsearch into bfloat16_esql_and_semantic_text_tests
@mromaios mromaios self-assigned this Dec 11, 2025
@Mikep86 Mikep86 added >enhancement :SearchOrg/Relevance Label for the Search (solution/org) Relevance team labels Dec 12, 2025
@Mikep86 Mikep86 requested a review from a team December 12, 2025 15:51
@elasticsearchmachine
Copy link
Collaborator

Hi @mromaios, I've created a changelog YAML for you.

@Mikep86 Mikep86 marked this pull request as ready for review December 12, 2025 15:54
@elasticsearchmachine elasticsearchmachine added the Team:Search - Relevance The Search organization Search Relevance team label Dec 12, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/search-relevance (Team:Search - Relevance)

Comment on lines +1595 to +1606
MapperService bitMapperService = mapperServiceForFieldWithModelSettings(
fieldName,
inferenceId,
new MinimalServiceSettings(
"my-service",
TaskType.TEXT_EMBEDDING,
1024,
SimilarityMeasure.L2_NORM,
DenseVectorFieldMapper.ElementType.BIT
)
);
assertMapperService.accept(bitMapperService, DenseVectorFieldMapper.ElementType.BIT);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We were missing this test case (in SemanticTextFieldMappterTests, at least), so I added it while I was already working in this area.

@Mikep86 Mikep86 requested a review from a team December 12, 2025 15:59
Copy link
Contributor

@jimczi jimczi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love the reactivity here but I don't think it's the right thing to do. We don't have models that return bfloat16 embeddings, they will all expose it as floats for now. The bfloat16 part is on the mapping side where models that return float are compatible with the element_type bfloat16. The idea is that models continue to generate floats but in the indexing we replace the floats with the bfloat16 transparently. So what we really need here is a way to change the element type in semantic text.
I'd also argue that we could make it the default for semantic text, maybe that should be the goal here, @benwtrent WDYT?

Copy link
Contributor

@jimczi jimczi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We chatted offline and agreed that it's ok to handle the new element type at the inference endpoint level. That will be useful for future models that handle this format natively but we shouldn't oblige users to change their inference endpoint where only a mapping change is needed. Embedding models that output floats should continue to use the float element type and semantic text fields should expose a way to change that granularly (opting for bfloat16 at index time).

@Mikep86 Mikep86 merged commit 8497d3d into elastic:main Dec 12, 2025
35 checks passed
parkertimmins pushed a commit to parkertimmins/elasticsearch that referenced this pull request Dec 17, 2025
Co-authored-by: Mike Pellegrini <mike.pellegrini@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>enhancement :SearchOrg/Relevance Label for the Search (solution/org) Relevance team Team:Search - Relevance The Search organization Search Relevance team v9.3.0

4 participants