Add refusal field to assistant conversations by jaybcee · Pull Request #243423 · elastic/kibana

jaybcee · 2025-11-18T21:44:47Z

Summary

capture the LLM refusal reason in inference chunk aggregation and langchain responses
persist messages.refusal in the assistant conversation schema/field map so it is searchable in the data stream
cover the new path with tests and keep helper types in sync

Related to https://github.com/elastic/search-team/issues/10868

Copilot

Pull request overview

This PR adds support for capturing and persisting LLM refusal reasons throughout the assistant conversation flow. When a model refuses to generate content (e.g., due to content filtering), the refusal reason is now captured from the LLM response, stored in the conversation data stream, and made searchable via the messages.refusal field.

Key changes:

Extended type definitions across the inference stack to include optional refusal field in messages, events, and API responses
Modified chunk aggregation logic to preserve refusal values from streaming responses
Updated conversation schema and field mappings to persist refusal data in Elasticsearch
Added LangChain adapter support to extract refusal from additional_kwargs

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
x-pack/platform/packages/shared/ai-infra/inference-common/src/chat_complete/messages.ts	Added optional `refusal` field to AssistantMessage type definition
x-pack/platform/packages/shared/ai-infra/inference-common/src/chat_complete/events.ts	Added optional `refusal` field to ChatCompletionMessageEvent and ChatCompletionChunkEvent
x-pack/platform/packages/shared/ai-infra/inference-common/src/chat_complete/api.ts	Added optional `refusal` field to ChatCompleteResponse interface
x-pack/platform/plugins/shared/inference/server/chat_complete/adapters/openai/from_openai.ts	Map OpenAI delta.refusal to chunk event's refusal field
x-pack/platform/plugins/shared/inference/server/chat_complete/utils/merge_chunks.ts	Aggregate refusal values from chunks into final message
x-pack/platform/plugins/shared/inference/server/chat_complete/utils/chunks_into_message.ts	Extract and conditionally include refusal in completion message event
x-pack/platform/plugins/shared/inference/server/chat_complete/utils/stream_to_response.ts	Pass refusal field through to response object
x-pack/platform/plugins/shared/stack_connectors/server/connector_types/inference/helpers.ts	Aggregate refusal from chunks in inference connector with initial null value
x-pack/platform/packages/shared/ai-infra/inference-langchain/src/chat_model/from_inference/chunks.ts	Add refusal to AIMessageChunk's additional_kwargs when present in chunks
x-pack/platform/packages/shared/ai-infra/inference-langchain/src/chat_model/from_inference/messages.ts	Convert inference response refusal to LangChain AIMessage additional_kwargs
x-pack/solutions/security/plugins/elastic_assistant/server/lib/langchain/executors/types.ts	Added optional refusal parameter to OnLlmResponse callback type
x-pack/solutions/security/plugins/elastic_assistant/server/lib/langchain/graphs/default_assistant_graph/helpers.ts	Extract refusal from AIMessage additional_kwargs in both streaming and invoke modes, pass to onLlmResponse
x-pack/solutions/security/plugins/elastic_assistant/server/routes/helpers.ts	Added messageRefusal parameter to appendAssistantMessageToConversation and getMessageFromRawResponse functions
x-pack/solutions/security/plugins/elastic_assistant/server/routes/post_actions_connector_execute.ts	Pass refusal from LLM response to message persistence logic
x-pack/solutions/security/plugins/elastic_assistant/server/routes/chat/chat_complete_route.ts	Pass refusal from LLM response to message persistence via onLlmResponse callback
x-pack/solutions/security/plugins/elastic_assistant/server/ai_assistant_data_clients/conversations/field_maps_configuration.ts	Added messages.refusal field mapping as non-required text field for searchability
x-pack/solutions/security/plugins/elastic_assistant/server/ai_assistant_data_clients/conversations/types.ts	Added optional refusal field to EsConversationSchema and CreateMessageSchema interfaces
x-pack/solutions/security/plugins/elastic_assistant/server/ai_assistant_data_clients/conversations/update_conversation.ts	Added optional refusal field to UpdateConversationSchema and conditionally include in transformed messages
x-pack/solutions/security/plugins/elastic_assistant/server/ai_assistant_data_clients/conversations/transforms.ts	Conditionally include refusal when transforming ES documents to conversation responses
x-pack/solutions/security/plugins/elastic_assistant/server/ai_assistant_data_clients/conversations/create_conversation.ts	Conditionally include refusal field when creating new conversation messages
x-pack/solutions/security/plugins/elastic_assistant/server/ai_assistant_data_clients/conversations/append_conversation_messages.ts	Conditionally include refusal field when appending messages to conversations
x-pack/solutions/security/plugins/elastic_assistant/server/ai_assistant_data_clients/conversations/append_conversation_messages.test.ts	Added test to verify refusal reason is preserved when present on messages
x-pack/platform/packages/shared/kbn-elastic-assistant-common/impl/schemas/conversations/common_attributes.schema.yaml	Added optional refusal field to Message schema with description
x-pack/platform/packages/shared/kbn-elastic-assistant-common/impl/schemas/conversations/common_attributes.gen.ts	Generated Zod schema with optional refusal string field for Message type

...tform/packages/shared/ai-infra/inference-langchain/src/chat_model/from_inference/messages.ts

+  const additionalKwargs = response.refusal ? { refusal: response.refusal } : undefined;
  return new AIMessage({
    content: response.content,
+    ...(additionalKwargs ? { additional_kwargs: additionalKwargs } : {}),


...ity/plugins/elastic_assistant/server/lib/langchain/graphs/default_assistant_graph/helpers.ts

+          const refusal =
+            typeof data.output?.additional_kwargs?.refusal === 'string'
+              ? (data.output.additional_kwargs.refusal as string)
+              : undefined;
+          handleFinalContent({ finalResponse: data.output.content, refusal, isError: false });


x-pack/solutions/security/plugins/elastic_assistant/server/routes/helpers.ts

    return {
      role: 'assistant',
      content: rawContent,
+      ...(refusal ? { refusal } : {}),


x-pack/platform/plugins/shared/inference/server/chat_complete/utils/chunks_into_message.ts

          return {
            type: ChatCompletionEventType.ChatCompletionMessage,
            content,
+            ...(refusal ? { refusal } : {}),


...ity/plugins/elastic_assistant/server/lib/langchain/graphs/default_assistant_graph/helpers.ts

+    const refusal =
+      typeof lastMessage?.additional_kwargs?.refusal === 'string'
+        ? (lastMessage.additional_kwargs.refusal as string)
+        : undefined;
    if (onLlmResponse) {
      await onLlmResponse({
        content: output,
        traceData,
+        ...(refusal ? { refusal } : {}),


x-pack/platform/plugins/shared/inference/server/chat_complete/utils/merge_chunks.ts

+      if (chunk.refusal) {
+        prev.refusal = chunk.refusal;
+      }


x-pack/platform/plugins/shared/inference/server/chat_complete/adapters/openai/from_openai.ts

  return {
    type: ChatCompletionEventType.ChatCompletionChunk,
    content: delta.content ?? '',
+    refusal: delta.refusal ?? undefined,


x-pack/platform/plugins/shared/stack_connectors/server/connector_types/inference/helpers.ts

+              if (chunk.choices[0].message.refusal) {
+                prev.choices[0].message.refusal = chunk.choices[0].message.refusal;
+              }


...latform/packages/shared/ai-infra/inference-langchain/src/chat_model/from_inference/chunks.ts

+  const additionalKwargs = chunk.refusal ? { refusal: chunk.refusal } : {};
+
  return new AIMessageChunk({
    content: chunk.content,
    tool_call_chunks: toolCallChunks,
-    additional_kwargs: {},
+    additional_kwargs: additionalKwargs,


kibanamachine · 2025-12-16T17:12:17Z

Project deployed, see credentials at: https://buildkite.com/elastic/kibana-deploy-project-from-pr/builds/816

kibanamachine · 2025-12-16T20:48:18Z

Cloud deployment initiated, see credentials at: https://buildkite.com/elastic/kibana-deploy-cloud-from-pr/builds/614

elasticmachine · 2025-12-18T19:34:00Z

💔 Build Failed

Buildkite Build
Commit: 1e3cc9c
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-243423-1e3cc9c43318
Security Deployment

Failed CI Steps

Test Failures

[job] [logs] FTR Configs #76 / Agent Builder agents Edit agent should edit agent name

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	10.8MB	10.8MB	+27.0B

Unknown metric groups

API count

id	before	after	diff
`@kbn/inference-common`	341	342	+1

History

💔 Build #375134 failed 80ba408
💛 Build #374718 was flaky e7d8437
💔 Build #374708 failed 0334219
💔 Build #374701 failed f72a1bc

qn895 · 2025-12-19T15:36:38Z

...ity/plugins/elastic_assistant/server/lib/langchain/graphs/default_assistant_graph/helpers.ts

-          handleFinalContent({ finalResponse: data.output.content, isError: false });
+          const refusal =
+            typeof data.output?.additional_kwargs?.refusal === 'string'
+              ? (data.output.additional_kwargs.refusal as string)


Do we need as string here?

qn895

Left a nit comment but LGTM 🎉

kibanamachine · 2025-12-19T15:40:08Z

Starting backport for target branches: 8.19, 9.1, 9.2, 9.3

https://github.com/elastic/kibana/actions/runs/20374815667

(cherry picked from commit fe9c9fd)

kibanamachine · 2025-12-19T15:48:28Z

💔 Some backports could not be created

Status	Branch	Result
❌	8.19	Backport failed because of merge conflicts
❌	9.1	Backport failed because of merge conflicts
❌	9.2	Backport failed because of merge conflicts
✅	9.3

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 243423

Questions ?

Please refer to the Backport tool documentation

(cherry picked from commit fe9c9fd) # Conflicts: # x-pack/solutions/security/plugins/elastic_assistant/server/lib/langchain/executors/types.ts # x-pack/solutions/security/plugins/elastic_assistant/server/lib/langchain/graphs/default_assistant_graph/helpers.ts # x-pack/solutions/security/plugins/elastic_assistant/server/routes/chat/chat_complete_route.ts # x-pack/solutions/security/plugins/elastic_assistant/server/routes/post_actions_connector_execute.ts

(cherry picked from commit fe9c9fd) # Conflicts: # x-pack/platform/packages/shared/ai-infra/inference-common/src/chat_complete/events.ts # x-pack/platform/packages/shared/ai-infra/inference-common/src/chat_complete/messages.ts # x-pack/solutions/security/plugins/elastic_assistant/server/ai_assistant_data_clients/conversations/append_conversation_messages.test.ts # x-pack/solutions/security/plugins/elastic_assistant/server/lib/langchain/executors/types.ts # x-pack/solutions/security/plugins/elastic_assistant/server/lib/langchain/graphs/default_assistant_graph/helpers.ts # x-pack/solutions/security/plugins/elastic_assistant/server/routes/chat/chat_complete_route.ts # x-pack/solutions/security/plugins/elastic_assistant/server/routes/post_actions_connector_execute.ts

stephmilovic · 2025-12-19T16:39:02Z

💚 All backports created successfully

Status	Branch	Result
✅	9.2
✅	9.1
✅	8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

(cherry picked from commit fe9c9fd) # Conflicts: # x-pack/platform/packages/shared/ai-infra/inference-common/src/chat_complete/events.ts # x-pack/platform/packages/shared/ai-infra/inference-common/src/chat_complete/messages.ts # x-pack/solutions/security/plugins/elastic_assistant/server/ai_assistant_data_clients/conversations/append_conversation_messages.test.ts # x-pack/solutions/security/plugins/elastic_assistant/server/lib/langchain/executors/types.ts # x-pack/solutions/security/plugins/elastic_assistant/server/lib/langchain/graphs/default_assistant_graph/helpers.ts # x-pack/solutions/security/plugins/elastic_assistant/server/routes/chat/chat_complete_route.ts # x-pack/solutions/security/plugins/elastic_assistant/server/routes/post_actions_connector_execute.ts

# Backport This will backport the following commits from `main` to `8.19`: - [Add refusal field to assistant conversations (#243423)](#243423)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  --------- Co-authored-by: Jason Botzas-Coluni <44372106+jaybcee@users.noreply.github.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

# Backport This will backport the following commits from `main` to `9.1`: - [Add refusal field to assistant conversations (#243423)](#243423)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  --------- Co-authored-by: Jason Botzas-Coluni <44372106+jaybcee@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

# Backport This will backport the following commits from `main` to `9.2`: - [Add refusal field to assistant conversations (#243423)](#243423)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  --------- Co-authored-by: Jason Botzas-Coluni <44372106+jaybcee@users.noreply.github.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

# Backport This will backport the following commits from `main` to `9.3`: - [Add refusal field to assistant conversations (#243423)](#243423)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  Co-authored-by: Jason Botzas-Coluni <44372106+jaybcee@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

jaybcee added 2 commits November 18, 2025 16:44

Add refusal field to assistant conversations

97ca865

Merge remote-tracking branch 'origin/main' into feature/refusal-field

4148b8e

jaybcee requested a review from Copilot December 1, 2025 19:24

Copilot started reviewing on behalf of jaybcee December 1, 2025 19:25 View session

Copilot finished reviewing on behalf of jaybcee December 1, 2025 19:28

Copilot AI reviewed Dec 1, 2025

View reviewed changes

jaybcee added 2 commits December 11, 2025 20:01

Merge branch 'main' into feature/refusal-field

cb52af2

Merge branch 'main' into feature/refusal-field

e245520

jaybcee added the ci:project-deploy-security Create a Security Serverless Project label Dec 16, 2025

jaybcee added ci:cloud-deploy Create or update a Cloud deployment ci:cloud-redeploy Always create a new Cloud deployment labels Dec 16, 2025

jaybcee marked this pull request as ready for review December 17, 2025 18:45

jaybcee requested review from a team as code owners December 17, 2025 18:45

Merge branch 'main' into feature/refusal-field

1e3cc9c

stephmilovic added v9.4.0 and removed v9.3.0 labels Dec 18, 2025

qn895 reviewed Dec 19, 2025

View reviewed changes

qn895 approved these changes Dec 19, 2025

View reviewed changes

stephmilovic merged commit fe9c9fd into main Dec 19, 2025
17 checks passed

stephmilovic deleted the feature/refusal-field branch December 19, 2025 15:39

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Dec 19, 2025

Add refusal field to assistant conversations (elastic#243423)

e7284c9

(cherry picked from commit fe9c9fd)

kibanamachine mentioned this pull request Dec 19, 2025

[9.3] Add refusal field to assistant conversations (#243423) #247135

Merged

stephmilovic mentioned this pull request Dec 19, 2025

[9.2] Add refusal field to assistant conversations (#243423) #247145

Merged

stephmilovic mentioned this pull request Dec 19, 2025

[9.1] Add refusal field to assistant conversations (#243423) #247148

Merged

stephmilovic mentioned this pull request Dec 19, 2025

[8.19] Add refusal field to assistant conversations (#243423) #247150

Merged

This was referenced Dec 19, 2025

[Security Solution] Updates kibana MITRE data to v18.1 #246770

Merged

[Security Solution][Entity Analytics][Risk Scoring] Handle special characters in ESQL query for risk scoring #247060

Merged

kibanamachine added the v8.19.10 label Dec 22, 2025

kibanamachine added the v9.1.10 label Dec 22, 2025

kibanamachine added the v9.2.4 label Dec 22, 2025

kibanamachine added the v9.3.0 label Dec 23, 2025

CAWilson94 pushed a commit to CAWilson94/kibana that referenced this pull request Jan 6, 2026

Add refusal field to assistant conversations (elastic#243423)

2ad1210

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add refusal field to assistant conversations#243423

Add refusal field to assistant conversations#243423
stephmilovic merged 9 commits intomainfrom
feature/refusal-field

jaybcee commented Nov 18, 2025 •

edited by kibanamachine

Loading

Copilot AI left a comment

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

kibanamachine commented Dec 16, 2025

kibanamachine commented Dec 16, 2025

elasticmachine commented Dec 18, 2025 •

edited

Loading

API count

qn895 Dec 19, 2025

qn895 left a comment

Uh oh!

kibanamachine commented Dec 19, 2025

kibanamachine commented Dec 19, 2025

stephmilovic commented Dec 19, 2025

Labels

7 participants

Conversation

jaybcee commented Nov 18, 2025 • edited by kibanamachine Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

kibanamachine commented Dec 16, 2025

kibanamachine commented Dec 16, 2025

elasticmachine commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💔 Build Failed

Failed CI Steps

Test Failures

Metrics [docs]

Async chunks

API count

History

qn895 Dec 19, 2025

Choose a reason for hiding this comment

qn895 left a comment

Choose a reason for hiding this comment

Uh oh!

kibanamachine commented Dec 19, 2025

kibanamachine commented Dec 19, 2025

💔 Some backports could not be created

Manual backport

Questions ?

stephmilovic commented Dec 19, 2025

💚 All backports created successfully

Questions ?

Labels

7 participants

jaybcee commented Nov 18, 2025 •

edited by kibanamachine

Loading

elasticmachine commented Dec 18, 2025 •

edited

Loading