refactor(semantic_text): fail early in pre-8.11 indices by mromaios · Pull Request #133080 · elastic/elasticsearch

mromaios · 2025-08-18T16:28:17Z

Reference Issue: #132551

In this PR we:

raise a mapper parsing exception when trying to create a semantic_text field in older indices. (Indices created prior to 8.11.0)

This happens for consistency as for

dense_vector fields, incompatible defaults are set including index=false
sparse_vector fields, were not supported in 8.0-8.10 and not indexed in 7.x.

elasticsearchmachine · 2025-08-18T16:30:43Z

Hi @mromaios, I've created a changelog YAML for you.

Mikep86

Looks to be on the right path to me! Testing will reveal whether the deprecation logger is emitting a warning header like we want to in this case.

...nference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java

…aios/elasticsearch into 132551_enable_dense_vector_indexing

...nference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java

elasticsearchmachine · 2025-08-19T14:38:48Z

Pinging @elastic/search-eng (Team:SearchOrg)

elasticsearchmachine · 2025-08-19T14:38:48Z

Pinging @elastic/search-relevance (Team:Search - Relevance)

…aios/elasticsearch into 132551_enable_dense_vector_indexing

* fix(semantic_text): index underlying dense_vector field in older indices * Update docs/changelog/133080.yaml * [CI] Auto commit changes from spotless * update msg, change versions, add ut * indent * remove todo * [CI] Auto commit changes from spotless * update msg * [CI] Auto commit changes from spotless * add densevectormapper indexed ut * [CI] Auto commit changes from spotless * fix ut * undo accidental ut removal * [CI] Auto commit changes from spotless * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * remove sparse_vector exception change * [CI] Auto commit changes from spotless * reverting most of the stuff * removing unused import, rename test * [CI] Auto commit changes from spotless * syntax * revisit changelog * revisit changelog * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * changelog format * fix UT --------- Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co> Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co>

elasticsearchmachine · 2025-08-26T18:20:09Z

💔 Backport failed

Status	Branch	Result
✅	9.1
❌	9.0	Commit could not be cherrypicked due to conflicts
❌	8.18	Commit could not be cherrypicked due to conflicts
❌	8.19	Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 133080

* fix(semantic_text): index underlying dense_vector field in older indices * Update docs/changelog/133080.yaml * [CI] Auto commit changes from spotless * update msg, change versions, add ut * indent * remove todo * [CI] Auto commit changes from spotless * update msg * [CI] Auto commit changes from spotless * add densevectormapper indexed ut * [CI] Auto commit changes from spotless * fix ut * undo accidental ut removal * [CI] Auto commit changes from spotless * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * remove sparse_vector exception change * [CI] Auto commit changes from spotless * reverting most of the stuff * removing unused import, rename test * [CI] Auto commit changes from spotless * syntax * revisit changelog * revisit changelog * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * changelog format * fix UT --------- Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co> Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> (cherry picked from commit 8f41a4b) # Conflicts: # server/src/test/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapperTests.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapperTests.java

* fix(semantic_text): index underlying dense_vector field in older indices * Update docs/changelog/133080.yaml * [CI] Auto commit changes from spotless * update msg, change versions, add ut * indent * remove todo * [CI] Auto commit changes from spotless * update msg * [CI] Auto commit changes from spotless * add densevectormapper indexed ut * [CI] Auto commit changes from spotless * fix ut * undo accidental ut removal * [CI] Auto commit changes from spotless * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * remove sparse_vector exception change * [CI] Auto commit changes from spotless * reverting most of the stuff * removing unused import, rename test * [CI] Auto commit changes from spotless * syntax * revisit changelog * revisit changelog * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * changelog format * fix UT --------- Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co> Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> (cherry picked from commit 8f41a4b) # Conflicts: # server/src/test/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapperTests.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java

kderusso · 2025-08-26T19:13:39Z

💚 All backports created successfully

Status	Branch	Result
✅	9.0
✅	8.19
✅	8.18

Questions ?

Please refer to the Backport tool documentation

* fix(semantic_text): index underlying dense_vector field in older indices * Update docs/changelog/133080.yaml * [CI] Auto commit changes from spotless * update msg, change versions, add ut * indent * remove todo * [CI] Auto commit changes from spotless * update msg * [CI] Auto commit changes from spotless * add densevectormapper indexed ut * [CI] Auto commit changes from spotless * fix ut * undo accidental ut removal * [CI] Auto commit changes from spotless * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * remove sparse_vector exception change * [CI] Auto commit changes from spotless * reverting most of the stuff * removing unused import, rename test * [CI] Auto commit changes from spotless * syntax * revisit changelog * revisit changelog * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * changelog format * fix UT --------- Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co> Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> (cherry picked from commit 8f41a4b) # Conflicts: # server/src/test/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapperTests.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapperTests.java

…33577) * fix(semantic_text): index underlying dense_vector field in older indices * Update docs/changelog/133080.yaml * [CI] Auto commit changes from spotless * update msg, change versions, add ut * indent * remove todo * [CI] Auto commit changes from spotless * update msg * [CI] Auto commit changes from spotless * add densevectormapper indexed ut * [CI] Auto commit changes from spotless * fix ut * undo accidental ut removal * [CI] Auto commit changes from spotless * Update docs/changelog/133080.yaml * remove sparse_vector exception change * [CI] Auto commit changes from spotless * reverting most of the stuff * removing unused import, rename test * [CI] Auto commit changes from spotless * syntax * revisit changelog * revisit changelog * Update docs/changelog/133080.yaml * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java * changelog format * fix UT --------- Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co> Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co>

… (#133578) * refactor(semantic_text): fail early in pre-8.11 indices (#133080) * fix(semantic_text): index underlying dense_vector field in older indices * Update docs/changelog/133080.yaml * [CI] Auto commit changes from spotless * update msg, change versions, add ut * indent * remove todo * [CI] Auto commit changes from spotless * update msg * [CI] Auto commit changes from spotless * add densevectormapper indexed ut * [CI] Auto commit changes from spotless * fix ut * undo accidental ut removal * [CI] Auto commit changes from spotless * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * remove sparse_vector exception change * [CI] Auto commit changes from spotless * reverting most of the stuff * removing unused import, rename test * [CI] Auto commit changes from spotless * syntax * revisit changelog * revisit changelog * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * changelog format * fix UT --------- Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co> Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> (cherry picked from commit 8f41a4b) # Conflicts: # server/src/test/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapperTests.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapperTests.java * [CI] Auto commit changes from spotless --------- Co-authored-by: Michail Romaios <84708293+mromaios@users.noreply.github.com> Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>

…) (#133580) * refactor(semantic_text): fail early in pre-8.11 indices (#133080) * fix(semantic_text): index underlying dense_vector field in older indices * Update docs/changelog/133080.yaml * [CI] Auto commit changes from spotless * update msg, change versions, add ut * indent * remove todo * [CI] Auto commit changes from spotless * update msg * [CI] Auto commit changes from spotless * add densevectormapper indexed ut * [CI] Auto commit changes from spotless * fix ut * undo accidental ut removal * [CI] Auto commit changes from spotless * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * remove sparse_vector exception change * [CI] Auto commit changes from spotless * reverting most of the stuff * removing unused import, rename test * [CI] Auto commit changes from spotless * syntax * revisit changelog * revisit changelog * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * changelog format * fix UT --------- Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co> Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> (cherry picked from commit 8f41a4b) # Conflicts: # server/src/test/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapperTests.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java * Fix test --------- Co-authored-by: Michail Romaios <84708293+mromaios@users.noreply.github.com>

…) (#133582) * refactor(semantic_text): fail early in pre-8.11 indices (#133080) * fix(semantic_text): index underlying dense_vector field in older indices * Update docs/changelog/133080.yaml * [CI] Auto commit changes from spotless * update msg, change versions, add ut * indent * remove todo * [CI] Auto commit changes from spotless * update msg * [CI] Auto commit changes from spotless * add densevectormapper indexed ut * [CI] Auto commit changes from spotless * fix ut * undo accidental ut removal * [CI] Auto commit changes from spotless * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * remove sparse_vector exception change * [CI] Auto commit changes from spotless * reverting most of the stuff * removing unused import, rename test * [CI] Auto commit changes from spotless * syntax * revisit changelog * revisit changelog * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * changelog format * fix UT --------- Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co> Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> (cherry picked from commit 8f41a4b) # Conflicts: # server/src/test/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapperTests.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapperTests.java * [CI] Auto commit changes from spotless * Fix test --------- Co-authored-by: Michail Romaios <84708293+mromaios@users.noreply.github.com> Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>

…ic#133080) (elastic#133580) * refactor(semantic_text): fail early in pre-8.11 indices (elastic#133080) * fix(semantic_text): index underlying dense_vector field in older indices * Update docs/changelog/133080.yaml * [CI] Auto commit changes from spotless * update msg, change versions, add ut * indent * remove todo * [CI] Auto commit changes from spotless * update msg * [CI] Auto commit changes from spotless * add densevectormapper indexed ut * [CI] Auto commit changes from spotless * fix ut * undo accidental ut removal * [CI] Auto commit changes from spotless * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * remove sparse_vector exception change * [CI] Auto commit changes from spotless * reverting most of the stuff * removing unused import, rename test * [CI] Auto commit changes from spotless * syntax * revisit changelog * revisit changelog * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> * changelog format * fix UT --------- Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co> Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co> (cherry picked from commit 8f41a4b) # Conflicts: # server/src/test/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapperTests.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java * Fix test --------- Co-authored-by: Michail Romaios <84708293+mromaios@users.noreply.github.com>

…tic#133080)" This reverts commit 8f41a4b.

fix(semantic_text): index underlying dense_vector field in older indices

4e4fae2

elasticsearchmachine added the v9.2.0 label Aug 18, 2025

mromaios self-assigned this Aug 18, 2025

mromaios added :SearchOrg/Relevance Label for the Search (solution/org) Relevance team >bug labels Aug 18, 2025

mromaios and others added 2 commits August 18, 2025 18:30

Update docs/changelog/133080.yaml

e63a078

[CI] Auto commit changes from spotless

a8fd795

Mikep86 reviewed Aug 18, 2025

View reviewed changes

...nference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Outdated Show resolved Hide resolved

mromaios and others added 15 commits August 19, 2025 15:23

update msg, change versions, add ut

a6eb387

Merge branch '132551_enable_dense_vector_indexing' of github.com:mrom…

fe8e551

…aios/elasticsearch into 132551_enable_dense_vector_indexing

indent

587726c

remove todo

46d0f47

[CI] Auto commit changes from spotless

db05915

Merge branch '132551_enable_dense_vector_indexing' of github.com:mrom…

13c713c

…aios/elasticsearch into 132551_enable_dense_vector_indexing

update msg

d8b18ac

[CI] Auto commit changes from spotless

42fa220

Merge branch '132551_enable_dense_vector_indexing' of github.com:mrom…

c2cef25

…aios/elasticsearch into 132551_enable_dense_vector_indexing

add densevectormapper indexed ut

d572fd7

[CI] Auto commit changes from spotless

9c48f54

Merge branch '132551_enable_dense_vector_indexing' of github.com:mrom…

ffa865c

…aios/elasticsearch into 132551_enable_dense_vector_indexing

fix ut

7fa6386

undo accidental ut removal

f0ab1e1

[CI] Auto commit changes from spotless

d5219ed

mromaios commented Aug 19, 2025

View reviewed changes

...nference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Outdated Show resolved Hide resolved

...nference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Outdated Show resolved Hide resolved

mromaios marked this pull request as ready for review August 19, 2025 14:38

elasticsearchmachine added Team:SearchOrg Meta label for the Search Org (Enterprise Search) Team:Search - Relevance The Search organization Search Relevance team labels Aug 19, 2025

mromaios added 2 commits August 22, 2025 09:29

Merge branch '132551_enable_dense_vector_indexing' of github.com:mrom…

2e262b3

…aios/elasticsearch into 132551_enable_dense_vector_indexing

fix UT

166964f

This was referenced Aug 22, 2025

docs(semantic_text): add index version limitation #133373

Merged

[8.19] docs(semantic_text): add index version limitation #133374

Merged

kderusso added v9.1.4 v9.0.7 auto-backport Automatically create backport pull requests when merged v8.19.4 v8.18.7 labels Aug 26, 2025

kderusso merged commit 8f41a4b into elastic:main Aug 26, 2025
34 checks passed

mromaios mentioned this pull request Aug 26, 2025

[9.1] refactor(semantic_text): fail early in pre-8.11 indices (#133080) #133577

Merged

elasticsearchmachine added the backport pending label Aug 26, 2025

This was referenced Aug 26, 2025

[9.0] refactor(semantic_text): fail early in pre-8.11 indices (#133080) #133578

Merged

[8.19] refactor(semantic_text): fail early in pre-8.11 indices (#133080) #133580

Merged

kderusso mentioned this pull request Aug 26, 2025

[8.18] refactor(semantic_text): fail early in pre-8.11 indices (#133080) #133582

Merged

Mikep86 removed the backport pending label Oct 2, 2025

Mikep86 added a commit to Mikep86/elasticsearch that referenced this pull request Oct 2, 2025

Revert "refactor(semantic_text): fail early in pre-8.11 indices (elas…

d19ed5d

…tic#133080)" This reverts commit 8f41a4b.

Mikep86 mentioned this pull request Oct 2, 2025

Defer Semantic Text Failures on Pre-8.11 Indices #135845

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(semantic_text): fail early in pre-8.11 indices#133080

refactor(semantic_text): fail early in pre-8.11 indices#133080
kderusso merged 39 commits intoelastic:mainfrom
mromaios:132551_enable_dense_vector_indexing

mromaios commented Aug 18, 2025 •

edited

Loading

elasticsearchmachine commented Aug 18, 2025

Mikep86 left a comment

Uh oh!

Uh oh!

Uh oh!

elasticsearchmachine commented Aug 19, 2025

elasticsearchmachine commented Aug 19, 2025

Uh oh!

elasticsearchmachine commented Aug 26, 2025

kderusso commented Aug 26, 2025

Labels

4 participants

Conversation

mromaios commented Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

elasticsearchmachine commented Aug 18, 2025

Mikep86 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

elasticsearchmachine commented Aug 19, 2025

elasticsearchmachine commented Aug 19, 2025

Uh oh!

elasticsearchmachine commented Aug 26, 2025

💔 Backport failed

kderusso commented Aug 26, 2025

💚 All backports created successfully

Questions ?

Labels

4 participants

mromaios commented Aug 18, 2025 •

edited

Loading