Skip to content

feat(server): add --api-key bearer authentication - #305

Open
benwilson wants to merge 1 commit into
FlashML-org:mainfrom
benwilson:feat/api-key
Open

feat(server): add --api-key bearer authentication#305
benwilson wants to merge 1 commit into
FlashML-org:mainfrom
benwilson:feat/api-key

Conversation

@benwilson

Copy link
Copy Markdown

Summary

  • add an optional, vLLM-compatible --api-key to ft serve (also read from FREETOKEN_API_KEY when the flag is absent, so the key need not appear in ps)
  • with a key set, every route except /health requires Authorization: Bearer <key>; anything else is answered 401 with a WWW-Authenticate: Bearer challenge and an OpenAI-shaped error body
  • unset, the server behaves exactly as today
  • shell mode keeps working: the attached client is handed the server's key, and ft shell attaching to a running server reads FREETOKEN_API_KEY
  • one row in docs/cli.md

Closes #152.

Motivation

ft serve currently has no way to require credentials, so any deployment that publishes the port beyond loopback is open, and clients that insist on a key (the report in #152) cannot be pointed at it without a reverse proxy in front. This is the minimal native path: one flag, one middleware, default unchanged — the same shape as #230 takes for TLS.

Design notes

  • What is guarded: everything but /health. The route surface includes mutating control routes (/v1/admin/prepare-stop, /v1/cache/rebuild), request history (/v1/requests) and /generate outside /v1, so a /v1-prefix rule would leave holes. /health stays open because liveness probes (load balancers, Docker healthchecks) and the desktop app's load-progress polling cannot carry a header and it reveals only status.
  • Ordering: the middleware is registered after _record_request_middleware, so Starlette runs it first — a 401 never lands in the request ring or a handler. OPTIONS passes through (a CORS preflight carries no credentials); install_cors runs at startup and stays outermost, so it answers the preflight and decorates the 401 with CORS headers.
  • Compare: hmac.compare_digest; the scheme is case-insensitive, whitespace around the token is not part of it.
  • Out of scope: the daemon's /engine/* proxy does not yet forward a key to a serve it manages (it cannot launch one with --api-key today), so nothing changes there.

Testing

  • PYTHONPATH=python python -m pytest tests/server -q563 passed (535 before + 28 in tests/server/test_api_key.py), on an RTX 3090 box with the [accel] install at this base
  • new tests cover: flag / env / precedence / empty-flag rejection / shell-mode parsing; 401 for a missing, wrong, truncated, over-long, Basic or bare credential; 200 for the matching bearer (case-insensitive scheme); /health open; every other route (/v1/chat/completions, /v1/messages, /v1/responses, /v1/models, /v1/stats, /v1/requests, /v1/cache/rebuild, /v1/admin/prepare-stop, /generate) gated before its handler; OPTIONS not challenged; install_api_key arm/disarm; the shell client sending the bearer on its control-plane requests
  • ft serve --help lists the flag next to --host/--port

I checked open and closed PRs for an existing api-key / auth implementation and found none (#152 is the open request; #230 is the adjacent TLS flag).

Require `Authorization: Bearer <key>` on every route except /health when
`ft serve --api-key` (or FREETOKEN_API_KEY) is set; 401 with a
WWW-Authenticate challenge otherwise. Unset, nothing changes.

- The check is a middleware registered after the request-ring middleware,
  so it runs first: a rejected request never reaches a handler or the
  ring. OPTIONS passes through for CORS preflights; the CORS middleware
  installed at startup stays outermost. Constant-time compare.
- /health stays open: liveness probes and the desktop app's load-progress
  polling cannot carry a header and reveal only status.
- Shell mode keeps working: the attached client is handed the server's
  key; `ft shell` attaching to a running server reads FREETOKEN_API_KEY.
- The control-plane requests of the shell client now send the same bearer
  the OpenAI client already sends.

Closes FlashML-org#152
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant