Skip to content

Route Anthropic models on Vertex AI through the native endpoint#2476

Merged
dgageot merged 1 commit intodocker:mainfrom
dgageot:board/fix-docker-agent-issue-2469-021fcb0c
Apr 24, 2026
Merged

Route Anthropic models on Vertex AI through the native endpoint#2476
dgageot merged 1 commit intodocker:mainfrom
dgageot:board/fix-docker-agent-issue-2469-021fcb0c

Conversation

@dgageot
Copy link
Copy Markdown
Member

@dgageot dgageot commented Apr 21, 2026

Summary

Fixes #2469.

Claude models on Google Cloud's Vertex AI Model Garden do not support the OpenAI-compatible `/chat/completions` endpoint and fail with:

`FAILED_PRECONDITION: The deployed model does not support ChatCompletions.`

When `provider_opts.publisher == "anthropic"`, requests are now routed through the Anthropic-native `:rawPredict` / `:streamRawPredict` endpoints (with the `vertex-2023-10-16` body schema) using the `anthropic-sdk-go/vertex` subpackage, authenticated via Google Application Default Credentials. Other Model Garden publishers (`meta`, `mistral`, ...) continue to use the OpenAI-compatible path, and Gemini-on-Vertex is unchanged (it never enters this code path).

Example config

```yaml
models:
claude-on-vertex:
provider: google
model: claude-sonnet-4-20250514
provider_opts:
project: my-gcp-project
location: us-east5
publisher: anthropic
```

Routing

Config Path
`provider: google`, publisher unset or `google` `gemini.NewClient` (unchanged)
`provider: google`, `publisher: anthropic` `anthropic.NewVertexClient` ( `:streamRawPredict` ) — new
`provider: google`, `publisher: meta` / `mistral` / ... OpenAI-compatible endpoint (unchanged)

Changes

  • `pkg/model/provider/anthropic/vertex.go` (new): `NewVertexClient` constructor that builds an Anthropic SDK client wired through `vertex.WithCredentials`. Credentials are resolved up front via `google.FindDefaultCredentials` so a missing ADC returns a friendly error. Explicitly clears `ANTHROPIC_API_KEY` (see in-code comment — this is required, not cosmetic).
  • `pkg/model/provider/vertexai/modelgarden.go`: single `NewClient` entry point that dispatches on publisher and returns a small `Client` interface satisfied by both `anthropic.Client` and `openai.Client`. Shared `resolveProjectLocation` helper with URL-injection-safe validation (regex blocks `..`, `/`, uppercase, etc.).
  • `pkg/model/provider/provider.go`: one-line dispatch into `vertexai.NewClient`.
  • Tests: cover publisher extraction, project/location resolution (env-var fallback, `${VAR}` expansion, URL-injection attempts, uppercase rejection).
  • Docs: `docs/providers/google/index.md` explains the two endpoint paths.
  • `go.mod` / `go.sum`: transitive `google.golang.org/api` dependency picked up by `go mod tidy` from the `anthropic-sdk-go/vertex` import.

Validation

  • `go build ./...` — clean.
  • `mise lint` — 0 issues (`golangci-lint` + custom cop over 755 files + `go mod tidy --diff` all clean).
  • `mise test` — full suite passes, including 11 new test cases covering URL-injection attempts.

Assisted-By: docker-agent

Claude models on Google Cloud's Vertex AI Model Garden do not support the OpenAI-compatible /chat/completions endpoint and fail with: 'FAILED_PRECONDITION: The deployed model does not support ChatCompletions.'

When provider_opts.publisher == "anthropic", requests are now routed through the Anthropic-native :rawPredict / :streamRawPredict endpoints (with the vertex-2023-10-16 body schema) using the anthropic-sdk-go/vertex subpackage, authenticated via Google Application Default Credentials. Other Model Garden publishers (meta, mistral, ...) continue to use the OpenAI-compatible path, and Gemini-on-Vertex is unchanged (it never enters this code path).

- pkg/model/provider/anthropic/vertex.go: new NewVertexClient constructor.

- pkg/model/provider/vertexai/modelgarden.go: single NewClient entry that dispatches on publisher and returns a small Client interface satisfied by both anthropic.Client and openai.Client; shared resolveProjectLocation helper with URL-injection-safe validation.

- pkg/model/provider/provider.go: one-line dispatch into vertexai.NewClient.

- Tests: cover publisher extraction, project/location resolution (env-var fallback, ${VAR} expansion, URL-injection attempts, uppercase rejection).

- Docs: docs/providers/google/index.md explains the two endpoint paths.

Fixes docker#2469

Assisted-By: docker-agent
@dgageot dgageot requested a review from a team as a code owner April 21, 2026 08:03
@dgageot dgageot merged commit c0ca2ab into docker:main Apr 24, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants