Skip to content

feat(benchmark): add optional Atlas VLM provider - #205

Open
binyangzhu000-sudo wants to merge 1 commit into
SharpAI:masterfrom
binyangzhu000-sudo:codex/add-atlas-vlm-provider
Open

feat(benchmark): add optional Atlas VLM provider#205
binyangzhu000-sudo wants to merge 1 commit into
SharpAI:masterfrom
binyangzhu000-sudo:codex/add-atlas-vlm-provider

Conversation

@binyangzhu000-sudo

Copy link
Copy Markdown

Summary

  • add opt-in Atlas Cloud VLM execution for the home-security benchmark
  • require explicit paid confirmation and environment-only credentials
  • disable SDK retries for Atlas and allow VLM-only runs without a local LLM

Validation

  • npm ci --ignore-scripts (0 vulnerabilities)
  • npm test (32 assertions passed)
  • node --check scripts/run-benchmark.cjs
  • CLI help and fail-fast gate checks
  • exactly one live Atlas smoke returned HTTP 402 (insufficient balance); it was not retried

Safety

  • local Aegis remains the default path
  • Atlas is enabled only with --atlas-vlm --confirm-paid-atlas
  • credentials are read from environment variables and are not accepted as CLI arguments
Signed-off-by: binyangzhu000-sudo <224954946+binyangzhu000-sudo@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant