Add `max_batch_size` setting to EIS sparse service settings by dimitris-athanasiou · Pull Request #141185 · elastic/elasticsearch

dimitris-athanasiou · 2026-01-23T15:09:40Z

Adds a new setting max_batch_size to the sparse service settings for EIS. The setting allows values of [1, 512]. The default value remains 16.

Tweaking this setting can help users improve performance.

Relates #1592

Adds a new setting `max_batch_size` to the dense and sparse service settings for EIS. The setting allows values of [1, 512]. The default value remains `16`. Tweaking this setting can help users improve performance. Relates elastic#1592

elasticsearchmachine · 2026-01-23T15:16:47Z

Pinging @elastic/search-inference-team (Team:Search - Inference)

elasticsearchmachine · 2026-01-23T15:17:11Z

Hi @dimitris-athanasiou, I've created a changelog YAML for you.

jonathan-buttner

Thanks for the changes, just left a few questions and suggestions.

...ervices/elastic/sparseembeddings/ElasticInferenceServiceSparseEmbeddingsServiceSettings.java

...s/elastic/densetextembeddings/ElasticInferenceServiceDenseTextEmbeddingsServiceSettings.java

...st/java/org/elasticsearch/xpack/inference/services/elastic/ElasticInferenceServiceTests.java

…le batch

DonalEvans · 2026-01-23T20:08:43Z

Should the new max_batch_size service settings field also be included in the output of the ElasticInferenceService.createConfiguration() method? Right now we only include model_id and max_input_tokens there, but we should probably also have max_batch_size for sparse_embedding and text_embedding and similarity and dimensions for text_embedding. Maybe the latter two fields can be left to another PR to keep the scope of this one from expanding too much.

.../inference/services/elastic/ElasticInferenceServiceSparseEmbeddingsServiceSettingsTests.java

...s/elastic/densetextembeddings/ElasticInferenceServiceDenseTextEmbeddingsServiceSettings.java

...stic/densetextembeddings/ElasticInferenceServiceDenseTextEmbeddingsServiceSettingsTests.java

.../inference/services/elastic/ElasticInferenceServiceSparseEmbeddingsServiceSettingsTests.java

...stic/densetextembeddings/ElasticInferenceServiceDenseTextEmbeddingsServiceSettingsTests.java

dimitris-athanasiou · 2026-01-26T13:50:49Z

@DonalEvans Thank you for the review! I have addressed all of your points. I have added the max_batch_size to the configuration reported by the services API, not the other 2. I'd rather to follow up and add those in a separate PR.

dimitris-athanasiou · 2026-01-26T18:10:01Z

@jonathan-buttner @DonalEvans I have removed the setting for dense models. Let me know if it's still good to go.

elasticsearchmachine · 2026-01-27T09:04:03Z

💔 Backport failed

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 141185

…141185) Adds a new setting `max_batch_size` to the sparse service settings for EIS. The setting allows values of [1, 512]. The default value remains `16`. Tweaking this setting can help users improve performance. Relates elastic#1592 (cherry picked from commit 01ace39) # Conflicts: # server/src/main/resources/transport/upper_bounds/9.4.csv

…141185) Adds a new setting `max_batch_size` to the sparse service settings for EIS. The setting allows values of [1, 512]. The default value remains `16`. Tweaking this setting can help users improve performance. Relates elastic#1592 (cherry picked from commit 01ace39) # Conflicts: # server/src/main/resources/transport/upper_bounds/9.4.csv # x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/inference/InferenceUtils.java # x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/inference/InferenceUtilsTests.java # x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/analysis/VerifierTests.java # x-pack/plugin/inference/src/internalClusterTest/java/org/elasticsearch/xpack/inference/integration/ModelRegistryIT.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/elastic/ElasticInferenceService.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/elastic/authorization/ElasticInferenceServiceAuthorizationModel.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/elastic/ElasticInferenceServiceSparseEmbeddingsServiceSettingsTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/elastic/ElasticInferenceServiceTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/elastic/action/ElasticInferenceServiceActionCreatorTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/elastic/authorization/AuthorizationPollerTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/elastic/authorization/ElasticInferenceServiceAuthorizationModelTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/elastic/response/ElasticInferenceServiceAuthorizationResponseEntityTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/googlevertexai/embeddings/GoogleVertexAiEmbeddingsServiceSettingsTests.java

dimitris-athanasiou · 2026-01-27T14:39:26Z

💚 All backports created successfully

Status	Branch	Result
✅	9.3
✅	9.2
✅	8.19

Questions ?

Please refer to the Backport tool documentation

Relates elastic#141185

…41185) (#141335) Adds a new setting `max_batch_size` to the sparse service settings for EIS. The setting allows values of [1, 512]. The default value remains `16`. Tweaking this setting can help users improve performance. Relates #1592 (cherry picked from commit 01ace39)

…141185) (#141362) Adds a new setting `max_batch_size` to the sparse service settings for EIS. The setting allows values of [1, 512]. The default value remains `16`. Tweaking this setting can help users improve performance. Relates #1592 (cherry picked from commit 01ace39)

Relates elastic#141185

Relates #141185

…141185) Adds a new setting `max_batch_size` to the sparse service settings for EIS. The setting allows values of [1, 512]. The default value remains `16`. Tweaking this setting can help users improve performance. Relates elastic#1592

…41185) (#141341) Adds a new setting `max_batch_size` to the sparse service settings for EIS. The setting allows values of [1, 512]. The default value remains `16`. Tweaking this setting can help users improve performance. Relates #1592 (cherry picked from commit 01ace39)

Relates #141185

We want to allow batches to be up to `1024` in size. Relates elastic#141185

We want to allow batches to be up to `1024` in size. Relates #141185

We want to allow batches to be up to `1024` in size. Relates elastic#141185 (cherry picked from commit e7f0bdd)

…1537) We want to allow batches to be up to `1024` in size. Relates #141185 (cherry picked from commit e7f0bdd)

…1536) We want to allow batches to be up to `1024` in size. Relates #141185 (cherry picked from commit e7f0bdd)

…1535) We want to allow batches to be up to `1024` in size. Relates #141185 (cherry picked from commit e7f0bdd)

dimitris-athanasiou added >enhancement auto-backport Automatically create backport pull requests when merged :SearchOrg/Inference Label for the Search Inference team v9.1.9 v9.3.1 v9.4.0 v8.19.11 v9.2.5 labels Jan 23, 2026

[CI] Update transport version definitions

a4b853a

elasticsearchmachine added the Team:Search - Inference label Jan 23, 2026

Update docs/changelog/141185.yaml

3d45f2b

jonathan-buttner approved these changes Jan 23, 2026

View reviewed changes

dimitris-athanasiou and others added 3 commits January 23, 2026 17:49

Renames some tests and adds test with multiple chunks fitting in sing…

cb09b02

…le batch

Extract common EIS settings utils

051dd6f

[CI] Auto commit changes from spotless

55b97f6

DonalEvans reviewed Jan 23, 2026

View reviewed changes

Address review feedback

0e5ee59

dimitris-athanasiou added 2 commits January 26, 2026 15:59

Merge branch 'main' into max-batch-size-for-eis

fa3bc19

Fixing some tests

0ce47e2

DonalEvans approved these changes Jan 26, 2026

View reviewed changes

Remove max_batch_size from dense

8109645

dimitris-athanasiou changed the title ~~Add max_batch_size setting to EIS dense and sparse service settings~~ Jan 26, 2026

DonalEvans approved these changes Jan 26, 2026

View reviewed changes

jonathan-buttner approved these changes Jan 26, 2026

View reviewed changes

dimitris-athanasiou merged commit 01ace39 into elastic:main Jan 27, 2026
35 checks passed

elasticsearchmachine added the backport pending label Jan 27, 2026

dimitris-athanasiou removed the v9.1.9 label Jan 27, 2026

dimitris-athanasiou mentioned this pull request Jan 27, 2026

[9.3] Add max_batch_size setting to EIS sparse service settings (#141185) #141335

Merged

dimitris-athanasiou mentioned this pull request Jan 27, 2026

[9.2] Add max_batch_size setting to EIS sparse service settings (#141185) #141341

Merged

dimitris-athanasiou mentioned this pull request Jan 27, 2026

[8.19] Add max_batch_size setting to EIS sparse service settings (#141185) #141362

Merged

dimitris-athanasiou added a commit to dimitris-athanasiou/elasticsearch that referenced this pull request Jan 27, 2026

Clean up code leftovers from 141185

feccda8

Relates elastic#141185

dimitris-athanasiou mentioned this pull request Jan 27, 2026

Clean up code leftovers from 141185 #141363

Merged

dimitris-athanasiou added a commit to dimitris-athanasiou/elasticsearch that referenced this pull request Jan 27, 2026

Fix a bug introduced in the backport of elastic#141185

b5ea4b4

Relates elastic#141185

dimitris-athanasiou mentioned this pull request Jan 27, 2026

Fix a bug introduced in the backport of #141185 #141395

Merged

jonathan-buttner pushed a commit that referenced this pull request Jan 27, 2026

Fix a bug introduced in the backport of #141185 (#141395)

c33442d

Relates #141185

dimitris-athanasiou deleted the max-batch-size-for-eis branch January 28, 2026 08:41

dimitris-athanasiou added a commit that referenced this pull request Jan 28, 2026

Clean up code leftovers from 141185 (#141363)

e6f02b0

Relates #141185

dimitris-athanasiou added a commit to dimitris-athanasiou/elasticsearch that referenced this pull request Jan 29, 2026

Increase EIS sparse max_batch_size upper bound to 1024

1cac021

We want to allow batches to be up to `1024` in size. Relates elastic#141185

dimitris-athanasiou mentioned this pull request Jan 29, 2026

Increase EIS sparse max_batch_size upper bound to 1024 #141516

Merged

dimitris-athanasiou added a commit that referenced this pull request Jan 29, 2026

Increase EIS sparse max_batch_size upper bound to 1024 (#141516)

e7f0bdd

We want to allow batches to be up to `1024` in size. Relates #141185

dimitris-athanasiou added a commit that referenced this pull request Jan 29, 2026

Increase EIS sparse max_batch_size upper bound to 1024 (#141516) (#14…

36fec1d

…1537) We want to allow batches to be up to `1024` in size. Relates #141185 (cherry picked from commit e7f0bdd)

dimitris-athanasiou added a commit that referenced this pull request Jan 29, 2026

Increase EIS sparse max_batch_size upper bound to 1024 (#141516) (#14…

54f68ee

…1536) We want to allow batches to be up to `1024` in size. Relates #141185 (cherry picked from commit e7f0bdd)

dimitris-athanasiou added a commit that referenced this pull request Jan 30, 2026

Increase EIS sparse max_batch_size upper bound to 1024 (#141516) (#14…

afbe438

…1535) We want to allow batches to be up to `1024` in size. Relates #141185 (cherry picked from commit e7f0bdd)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `max_batch_size` setting to EIS sparse service settings#141185

Add `max_batch_size` setting to EIS sparse service settings#141185
dimitris-athanasiou merged 10 commits intoelastic:mainfrom
dimitris-athanasiou:max-batch-size-for-eis

dimitris-athanasiou commented Jan 23, 2026 •

edited

Loading

elasticsearchmachine commented Jan 23, 2026

elasticsearchmachine commented Jan 23, 2026

jonathan-buttner left a comment

Uh oh!

Uh oh!

Uh oh!

DonalEvans commented Jan 23, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dimitris-athanasiou commented Jan 26, 2026

dimitris-athanasiou commented Jan 26, 2026

Uh oh!

elasticsearchmachine commented Jan 27, 2026

dimitris-athanasiou commented Jan 27, 2026

Labels

4 participants

Conversation

dimitris-athanasiou commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

elasticsearchmachine commented Jan 23, 2026

elasticsearchmachine commented Jan 23, 2026

jonathan-buttner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

DonalEvans commented Jan 23, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dimitris-athanasiou commented Jan 26, 2026

dimitris-athanasiou commented Jan 26, 2026

Uh oh!

elasticsearchmachine commented Jan 27, 2026

💔 Backport failed

dimitris-athanasiou commented Jan 27, 2026

💚 All backports created successfully

Questions ?

Labels

4 participants

dimitris-athanasiou commented Jan 23, 2026 •

edited

Loading