[AI Infra] Adds new temperature parameter to AI Connector and OpenAI, Bedrock, Gemini connectors by qn895 · Pull Request #239806 · elastic/kibana

qn895 · 2025-10-20T18:51:09Z

Summary

Follow up of #239626. This PR addresses #239181.

If a user defines/sets a temperature at the connector level, the connector's temperature will override the Kibana's default temperature for each connector. If temperature is not explicitly set by a user, then we respect the the temperature set by the request.

AI connector

Before:

After:

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
Flaky Test Runner was used on any tests changed
The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
Review the backport guidelines and apply applicable backport:* labels.

Identify risks

Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.

See some risk examples
...

…emperature

alvarezmelissa87 · 2025-10-29T15:20:55Z

...ackages/shared/kbn-inference-endpoint-ui-common/src/components/additional_options_fields.tsx

@@ -338,6 +436,7 @@ export const AdditionalOptionsFields: React.FC<AdditionalOptionsFieldsProps> = (
        </UseField>
        <EuiSpacer size="m" />
        {contextWindowLengthSettings}


Is temperature something that can be set with every connector or just when using chat_completion task type?

Probably just chat_completion or completion 👀

Yeah I assume it only makes sense for those two task types

pgayvallet · 2025-10-30T13:10:39Z

...ackages/shared/kbn-inference-endpoint-ui-common/src/components/additional_options_fields.tsx

@@ -338,6 +436,7 @@ export const AdditionalOptionsFields: React.FC<AdditionalOptionsFieldsProps> = (
        </UseField>
        <EuiSpacer size="m" />
        {contextWindowLengthSettings}


Yeah I assume it only makes sense for those two task types

pgayvallet · 2025-10-30T13:13:13Z

...ackages/shared/kbn-inference-endpoint-ui-common/src/components/additional_options_fields.tsx

+                    if (isNaN(numValue) || numValue < 0 || numValue > 1) {
+                      return {
+                        code: 'ERR_FIELD_INVALID',
+                        path,
+                        message: LABELS.TEMPERATURE_VALIDATION_MESSAGE,
+                      };


I don't think temperature is capped to 1. Note that the max value may depend on the provider and model 🙈. But it's probably fine to let them pick whatever they want here. Worth case it breaks when using the connector with an explicit error (which is the case) and the customer can just edit it again.

Updated here 20650d7 (#239806)

pgayvallet · 2025-10-30T13:14:36Z

x-pack/platform/plugins/shared/inference/server/chat_complete/utils/get_temperature.ts

+    return { temperature: connector.config.temperature };
+  }
+
+  if (temperature === undefined || temperature < 0 || temperature > 1) return {};


Same remark about max value

Thanks! Updated here 20650d7 (#239806)

pgayvallet · 2025-10-30T13:35:07Z

x-pack/platform/plugins/shared/inference/server/chat_complete/utils/get_temperature.ts

+  // If user sets temperature in the connector config, use it by default
+  if (connector?.config?.temperature) {
+    return { temperature: connector.config.temperature };
+  }


So - the approach is fine for any usage of the GenAI connectors going through the inference plugin.

This new temperature config param will not be used when calling the connectors directly. I'm not sure this is a problem, given the inference plugin is the recommended way of calling LLMs from Kibana, but I just wanted to make sure we were aware of it.

Yeah I'm trying to support for those connectors not going through the inference plugin too for this PR but it's a bit time consuming, so maybe a follow up is fine. Thank you for bringing this up!

elasticmachine · 2025-10-30T15:25:17Z

Pinging @elastic/ml-ui (:ml)

elasticmachine · 2025-10-30T15:25:29Z

Pinging @elastic/appex-ai-infra (Team:AI Infra)

…emperature

alvarezmelissa87 · 2025-10-30T17:42:31Z

...ackages/shared/kbn-inference-endpoint-ui-common/src/components/additional_options_fields.tsx


+  const temperatureSettings = useMemo(
+    () =>
+      taskTypeOptions?.some((option) => option.id === CHAT_COMPLETION_TASK_TYPE) ||


Since this is only compatible with chat completion, then we should enforce that in the UI by only showing this setting if chat completion is selected.

Updated here 6989330 (#239806)

alvarezmelissa87 · 2025-10-30T18:27:35Z

Tested AI Connector (openAI provider) and confirmed temperature is sent along correctly.
Tested Open AI specific connector and confirmed temperature is sent along correctly.

stephmilovic

LGTM! Thanks for the improvement!

jcger · 2025-11-03T12:24:58Z

x-pack/platform/plugins/shared/stack_connectors/public/common/genai_connectors/index.tsx

 import { FormattedMessage } from '@kbn/i18n-react';
 import * as i18n from './translations';

+export const temperatureField: ConfigFieldSchema = {


no default value?

no,by design. Default value is meant to be undefined (defaulting to the provider's native default)

alvarezmelissa87 · 2025-11-05T16:43:08Z

@elasticmachine merge upstream

alvarezmelissa87

LGTM ⚡

elasticmachine · 2025-11-05T23:15:25Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 24a461d

Failed CI Steps

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/inference-endpoint-ui-common`	170	171	+1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`indexManagement`	698.4KB	700.8KB	+2.4KB
`searchInferenceEndpoints`	114.4KB	116.7KB	+2.4KB
`searchPlayground`	235.0KB	235.3KB	+277.0B
`securitySolution`	11.0MB	11.0MB	+277.0B
`stackConnectors`	725.5KB	730.4KB	+4.9KB
total			+10.2KB

Unknown metric groups

API count

id	before	after	diff
`@kbn/inference-endpoint-ui-common`	170	171	+1

History

💔 Build #358744 failed 09eb17a
💛 Build #358681 was flaky 1ac2079
💚 Build #356959 succeeded 132e112
💔 Build #356936 failed 6989330
💛 Build #356884 was flaky b76b3ab

cc @qn895

… Bedrock, Gemini connectors (elastic#239806) ## Summary Follow up of elastic#239626. This PR addresses elastic#239181. If a user defines/sets a temperature at the connector level, the connector's temperature will override the Kibana's default temperature for each connector. If temperature is not explicitly set by a user, then we respect the the temperature set by the request. AI connector <img width="866" height="880" alt="Screenshot 2025-10-28 at 09 16 29" src="https://github.com/user-attachments/assets/e5d28754-0b4d-46df-99b7-43c26bb9ab5a" /> <img width="866" height="880" alt="Screenshot 2025-10-28 at 09 20 18" src="https://github.com/user-attachments/assets/71e3a125-e8a3-4bd9-a04d-c34b29e0b7a1" /> <img width="866" height="880" alt="Screenshot 2025-10-28 at 09 18 18" src="https://github.com/user-attachments/assets/6fa62eb0-24dc-4245-bc3f-a84fa10b34c4" /> <img width="866" height="880" alt="Screenshot 2025-10-28 at 09 18 03" src="https://github.com/user-attachments/assets/b60391fd-f23f-40c5-b8a8-87c419f5b2c6" /> Before: <img width="1896" height="1386" alt="image" src="https://github.com/user-attachments/assets/6213ee44-d53e-440e-95c3-ac1abaded3c0" /> After: ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [ ] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Melissa <melissa.alvarez@elastic.co>

qn895 added 6 commits October 17, 2025 15:04

Add new temperature parameter to schema

d31df7a

Add temperature schema for legacy connectors too

0935d44

Add new temperature override from connector

433c025

Update snapshot

9bed7f3

Merge upstream/main

e89b425

Merge remote-tracking branch 'upstream/main' into ai-connectors-add-t…

25d0941

…emperature

qn895 changed the title ~~[AI Infra] Add new temperature parameter to GenAI Connectors (OpenAI, AI Connector, Gemini, and Bedrock)~~ Oct 27, 2025

qn895 added 4 commits October 27, 2025 16:29

Update for OpenAI and Inference connector

fd70da9

Update adapters

aa49b80

Merge remote-tracking branch 'upstream/main' into ai-connectors-add-t…

b685e63

…emperature

Update snapshots

6434694

qn895 marked this pull request as ready for review October 28, 2025 14:27

qn895 requested review from a team as code owners October 28, 2025 14:27

Add optional props

ec215cf

qn895 requested a review from alvarezmelissa87 October 28, 2025 17:42

Update jest tests

e763213

alvarezmelissa87 reviewed Oct 29, 2025

View reviewed changes

qn895 changed the title ~~[AI Infra] Add new temperature parameter to Inference GenAI Connector~~ Oct 29, 2025

pgayvallet reviewed Oct 30, 2025

View reviewed changes

peteharverson added the :ml label Oct 30, 2025

peteharverson added the Team:AI Infra Platform AppEx AI Infrastructure Team t// label Oct 30, 2025

peteharverson added the Feature: AI Infra Relating to the AI Assistant flow and any work impacting/involving the AI/Inference Connector label Oct 30, 2025

qn895 requested review from a team as code owners October 30, 2025 15:34

qn895 added the release_note:enhancement label Oct 30, 2025

qn895 self-assigned this Oct 30, 2025

qn895 added 3 commits October 30, 2025 11:08

Fix completion, max validation

20650d7

Merge remote-tracking branch 'upstream/main' into ai-connectors-add-t…

48ff7ca

…emperature

Fix duplicate temperature from merge

b76b3ab

alvarezmelissa87 reviewed Oct 30, 2025

View reviewed changes

qn895 added 2 commits October 30, 2025 13:36

Fix when to show

6989330

Fix linting and update messaging

132e112

stephmilovic approved these changes Oct 31, 2025

View reviewed changes

neptunian approved these changes Oct 31, 2025

View reviewed changes

jcger approved these changes Nov 3, 2025

View reviewed changes

Merge branch 'main' into ai-connectors-add-temperature

1ac2079

alvarezmelissa87 approved these changes Nov 5, 2025

View reviewed changes

alvarezmelissa87 added 2 commits November 5, 2025 13:17

only allow temperature to be set for connector

09eb17a

add missing allowTemperature dependency

24a461d

alvarezmelissa87 merged commit cf5ff5b into elastic:main Nov 5, 2025
13 checks passed

kibanamachine added backport:skip This PR does not require backporting v9.3.0 and removed backport:version Backport to applied version labels labels Nov 5, 2025

alvarezmelissa87 added v9.3.0 and removed v9.3.0 labels Nov 5, 2025

This was referenced Nov 18, 2025

[Automatic Import] Cannot use o3/o4-mini due to temperature #218693

Open

[Attack Discovery] Adds support for running Attack Discovery against models that do not support temperature #243430

Draft

peteharverson changed the title ~~[AI Infra] Add new temperature parameter to Inference AI, and OpenAI, Bedrock, Gemini connectors~~ Dec 8, 2025

qn895 mentioned this pull request Dec 18, 2025

[AI Connector] Allow temperature to be set on the connector level #239181

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[AI Infra] Adds new temperature parameter to AI Connector and OpenAI, Bedrock, Gemini connectors#239806

[AI Infra] Adds new temperature parameter to AI Connector and OpenAI, Bedrock, Gemini connectors#239806
alvarezmelissa87 merged 20 commits intoelastic:mainfrom
qn895:ai-connectors-add-temperature

qn895 commented Oct 20, 2025 •

edited

Loading

alvarezmelissa87 Oct 29, 2025

qn895 Oct 29, 2025

pgayvallet Oct 30, 2025

pgayvallet Oct 30, 2025

pgayvallet Oct 30, 2025

qn895 Oct 30, 2025

pgayvallet Oct 30, 2025

qn895 Oct 30, 2025

pgayvallet Oct 30, 2025

qn895 Oct 30, 2025

elasticmachine commented Oct 30, 2025

elasticmachine commented Oct 30, 2025

alvarezmelissa87 Oct 30, 2025

qn895 Oct 30, 2025

alvarezmelissa87 commented Oct 30, 2025 •

edited

Loading

stephmilovic left a comment

jcger Nov 3, 2025

pgayvallet Nov 5, 2025

alvarezmelissa87 commented Nov 5, 2025

alvarezmelissa87 left a comment

elasticmachine commented Nov 5, 2025

API count

Uh oh!

Labels

9 participants

Conversation

qn895 commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Checklist

Identify risks

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elasticmachine commented Oct 30, 2025

elasticmachine commented Oct 30, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alvarezmelissa87 commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

stephmilovic left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alvarezmelissa87 commented Nov 5, 2025

alvarezmelissa87 left a comment

Choose a reason for hiding this comment

elasticmachine commented Nov 5, 2025

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Public APIs missing comments

Async chunks

API count

History

Uh oh!

Labels

9 participants

qn895 commented Oct 20, 2025 •

edited

Loading

alvarezmelissa87 commented Oct 30, 2025 •

edited

Loading