[Serverless] Preconfigured Connectors: adds new connectors and updates existing one

[alvarezmelissa87]

Summary

Serverless and on prem kibana pull from these yaml files for the preconfigured connector configurations. ESS changes are made in the cloud repo. A PR has also been raised for that.

Related issue #243094
This PR is part of the work to support multiple managed LLMs.

This PR:

• Adds the "General Purpose LLM v2" which uses .gp-llm-v2-chat_completion as the inference id and "General Purpose LLM v3" which uses .gp-llm-v3-chat_completion as the inference id.
  • It should be okay to add before the inference endpoint is available as it will be filtered out of the connectors returned to Kibana if the endpoint does not exist. See relevant code.
  • Once the endpoint is available, the preconfigured connector will show up in the connectors list.
• Renames "Elastic Managed LLM" to "General Purpose LLM v1"
  • This change also required some updates in places that rely on the old name when checking if a connector is internal/preconfigured.
  • In those cases, both the new connector names have been added

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
• This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
• Flaky Test Runner was used on any tests changed
• The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
• Review the backport guidelines and apply applicable backport:* labels.

[elasticmachine]

Pinging @elastic/ml-ui (:ml)

[alvarezmelissa87]

/ci

[Samiul-TheSoccerFan]

Left few questions in the PR. Are the inference endpoints are also preconfigured or only the connectors?

[Samiul-TheSoccerFan]

Same question if we need Elastic-Managed-LLM in here and in the following file.

[pmuellr]

Ya, it looks like this PR removes Elastic-Managed-LLM and adds General-Purpose-LLM-v{1,2,3}.

Anyone who has a saved object that references Elastic-Managed-LLM won't have access to it after this change. Will be returned as "not found". I'm not sure what, if anything, might have a reference. I don't think these can be used directly as actions from alerting rules. If they can, then I think this will be a problem.

Or perhaps these connector id's are never referenced directly, just used internally?

[alvarezmelissa87]

@pmuellr - thanks for taking a look! 🙏
This updates the name of the preconfigured connector -Elastic-Managed-LLM to General-Purpose-LLM-v1 - the underlying inference endpoint and model are unchanged. I also updated the places in kibana that reference the connector id directly to look for both the old and new name. We've updated the naming/connector ids for preconfigured connectors in the past and it should not present a problem. The connector ids are internal - it's the underlying inference endpoint ids that are used.

The additional preconfigured connectors added (v2, v3) will only be returned by the actions client getAll once the backing inference endpoint exists, so no issue there either.

[pmuellr]

Sounds good. I'd go ahead and try testing an upgrade, if you haven't already, by running

yarn es serverless --license trial --projectType $PROJECT_TYPE

and then run Kibana from main to do something to get a reference to Elastic-Managed-LLM, then kill Kibana, then run the same from your PR to see if everything survived "migration".

yarn start --no-base-path --serverless $PROJECT_TYPE

[MichelLosier]

Looks good from Fleet!

[alvarezmelissa87]

@elasticmachine merge upstream

[tomsonpl]

Defend Workflows code changes LGTM 👍

[pmuellr]

ResponseOps changes LGTM. I noted some concerns here: #242791 (comment)

Sounds like the connector id's are never persisted anywhere (like other saved objects), so when the existing connector Elastic-Managed-LLM disappears when this PR is merged, nothing will break.

[seanstory]

We want Rainbow Sprinkles to be the "first" in this list, so that it's what's used by default for various AI tools. But I have no issue with adding more models.

[alvarezmelissa87]

Thank you for taking a look, @seanstory! 🙏

So in this PR I updated the places where the old name was being specifically checked for things like order/defaults to make sure both names are recognized (in case of timing issues with the cloud changes).

I will go through and ensure the default is unchanged for assistants/playground. Other entry points I may have missed will need to be addressed specifically.
I will also have a follow up to update all the copy that references the old name but was trying to keep this PR smaller and isolate the required changes more for better tracking.

The plan is to update default to v2 once this change is in and all teams are happy with changes. Aiming for before FF 9.3.

[seanstory]

To be clear, I don't care about the name. What I care about is the order of this list for xpack.actions.preconfigured.

A number of AI features in Kibana predate any settings/configurations for when you might have multiple LLM connectors. Instead, they do something naive, like llmConnection = listConnectors().filter(c -> c.type == 'llm')[0], so it's important that we keep the expected default LLM as the "first" in the list.

It's possible that I'm out of touch, and that the new v3/v2 LLMs are already what we want the "default" to be. If that's the case, I retract this comment thread.

[alvarezmelissa87]

Good point! To be safe - updated the order so it's v1, v2, v3 in 0146ed4

I'm dismissing my review so that I'm not blocking you. As long as someone from Inference is happy with this and how it changes the "default LLM connector" for AI assistants and Agent Builder, I'm sure it's fine. :)

[github-actions]

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

• /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[alvarezmelissa87]

/ci

[alvarezmelissa87]

/ci

[alvarezmelissa87]

@elasticmachine merge upstream

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 5b89335
• Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-242791-5b8933554b11
• Security Deployment
• Elasticsearch Serverless Deployment

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
securitySolution	11.1MB	11.1MB	+78.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
elasticAssistant	307.4KB	307.5KB	+41.0B

History

• 💔 Build #368535 failed f45d651
• 💔 Build #368017 failed f45d651
• 💔 Build #367933 failed f45d651
• 💚 Build #365386 succeeded 0146ed4

cc @alvarezmelissa87