Openrouter or custom model with Openapi compatible

[yamijuan]

Is there any way or plan to add open router as model provider or be able to set up a custom model provider that's openapi compatible? thanks

[mrT23]

We support a lot of different providers:

Openai-compatible providers (like vllm) are supported, among others, via litellm
https://qodo-merge-docs.qodo.ai/usage-guide/changing_a_model/#ollama

[user753]

@yamijuan Did you figure out how to use openrouter?

[yamijuan]

I haven't implemented it yet, but I read the documentation, and seems clear what to do with litellm

[mrT23]

if open-router (which i have not used) are providing openai-compatible mechanism, that it is fully supported with pr-agent

something like:

[openai]
custom_llm_provider ="openai"
api_base ="..."

[config]
model="openai/Qwen2.5-32B"
model_turbo="openai/Qwen2.5-32B"
fallback_models=["openai/Qwen2.5-32B"]

[Ocrosoft]

I encountered the following issue when using the third-party OpenAI interface.

[config]
model = "openai/gpt-4-0613"

This is part of the configuration file, and then I receive the following error:

pr_agent.algo.utils:clip_tokens:924 - Failed to clip tokens: 'Could not automatically map openai/gpt-4-0613 to a tokeniser. Please use `tiktoken.get_encoding` to explicitly get the tokeniser you expect.'

Perhaps we need to remove the "openai/" part when calling encoding_for_model?

[yamijuan]

if open-router (which i have not used) are providing openai-compatible mechanism, that it is fully supported with pr-agent

something like:

[openai]
custom_llm_provider ="openai"
api_base ="..."

[config]
model="openai/Qwen2.5-32B"
model_turbo="openai/Qwen2.5-32B"
fallback_models=["openai/Qwen2.5-32B"]

I followed this, while not the way litellm adds openrouter, this worked

[yamijuan]

I've struggled with this a lot, reached a point where routing is being done to open router but litellm fails to send auth credentials

[irfansofyana]

anyone know what config looks like if we use openrouter?

[irfansofyana]

after some struggle, I can found the rootcause and create this PR: #1744 it should solved the auth error

[vicky7230]

i am trying to use openrouter ai in my github ci. Below is my code:

name: CodiumAI PR-Agent

on:
  pull_request:
    types: [opened, reopened, ready_for_review, review_requested, synchronize]
  issue_comment:

jobs:
  pr_agent_job:
    if: ${{ github.event.sender.type != 'Bot' }}
    runs-on: ubuntu-latest
    permissions:
      issues: write
      pull-requests: write
      contents: write
    steps:
      - name: Checkout PR code
        uses: actions/checkout@v3

      - name: Run PR Agent
        id: pragent
        uses: qodo-ai/pr-agent@main
        env:
          OPENAI_KEY: ${{ secrets.OPENROUTER_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

here is my config file name : .pr_agent.toml

[config]
model="openrouter/qwen/qwen-2.5-coder-32b-instruct:free"
[github_action_config]
pr_actions = ['opened', 'reopened', 'ready_for_review', 'review_requested', 'synchronize']

My ci is failing with this error:

{"text": "Error during LLM inference: litellm.AuthenticationError: AuthenticationError: OpenAIException - Incorrect API key provided: sk-or-v1*************************************************************9352. You can find your API key at [https://platform.openai.com/account/api-keys.\n](https://platform.openai.com/account/api-keys./n)", "record": {"elapsed": {"repr": "0:00:08.026504", "seconds": 8.026504}, "exception": null, "extra": {}, "file": {"name": "litellm_ai_handler.py", "path": "/app/pr_agent/algo/ai_handlers/litellm_ai_handler.py"}, "function": "chat_completion", "level": {"icon": "⚠️", "name": "WARNING", "no": 30}, "line": 378, "message": "Error during LLM inference: litellm.AuthenticationError: AuthenticationError: OpenAIException - Incorrect API key provided: sk-or-v1*************************************************************9352. You can find your API key at https://platform.openai.com/account/api-keys.", "module": "litellm_ai_handler", "name": "pr_agent.algo.ai_handlers.litellm_ai_handler", "process": {"id": 7, "name": "MainProcess"}, "thread

[vicky7230]

@irfansofyana @yamijuan @mrT23 Can you help me here?

[mrT23]

OpenAIException - Incorrect API key provided

triple check your key

[vicky7230]

it was not the key issue. Key was fine. This is what finally worked:

ci pipelien code:

name: AI Pr Review

on:
  pull_request:
    types: [opened, reopened, ready_for_review, review_requested, synchronize]
  issue_comment:

jobs:
  pr_agent_job:
    name: Review Pr
    if: ${{ github.event.sender.type != 'Bot' }}
    runs-on: ubuntu-latest
    permissions:
      issues: write
      pull-requests: write
      contents: write
    steps:
      - name: Checkout PR code
        uses: actions/checkout@v3

      - name: Run PR Agent
        id: pragent
        uses: qodo-ai/pr-agent@main
        env:
          OPENROUTER__KEY: ${{ secrets.OPENROUTER__KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

config:

[config]
model="openrouter/qwen/qwen-2.5-coder-32b-instruct:free"
custom_model_max_tokens=4000
[github_action_config]
auto_review = true
auto_describe = true
auto_improve = true
pr_actions = ['opened', 'reopened', 'ready_for_review', 'review_requested', 'synchronize']

[wozniak-slawomir]

I have the same issue as @vicky7230. Could someone help me? I have no idea why it does not work. If we figure this out I may submit a PR with documentation about that.

My .github/workflows/pr-agent.yml:

name: PR Agent
on:
  push:
    branches: 
      - pr-test
  pull_request:
    types: [opened, reopened, ready_for_review]
  issue_comment:
jobs:
  pr_agent_job:
    if: ${{ github.event.sender.type != 'Bot' }}
    runs-on: ubuntu-latest
    permissions:
      issues: write
      pull-requests: write
      contents: write
    name: Run pr agent on every pull request, respond to user comments
    steps:
      - name: Checkout PR code
        uses: actions/checkout@v3
      - name: PR Agent action step
        id: pragent
        uses: qodo-ai/pr-agent@main
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          OPENROUTER__KEY: ${{ secrets.OPENROUTER_API_KEY }}
          # Tool configuration
          github_action_config.auto_review: "true"
          github_action_config.auto_describe: "true"
          github_action_config.auto_improve: "true"
          # Additional configuration for better reliability
          config.ai_timeout: "120"

My .pr_agent.toml:

model = "openrouter/qwen/qwen3-coder:free"
fallback_models = ["openrouter/deepseek/deepseek-chat"]
custom_model_max_tokens=20000

[pr_reviewer]
extra_instructions = "Focus on security issues and code quality."

[vicky7230]

your .pr_agent.toml file should look like this:

[config]
model="openrouter/qwen/qwen-2.5-coder-32b-instruct:free"
custom_model_max_tokens=4000
[github_action_config]
auto_review = true
auto_describe = true
auto_improve = true
pr_actions = ['opened', 'reopened', 'ready_for_review', 'review_requested', 'synchronize']

also try changing your key to secrets.OPENROUTER__KEY in your github repo secrets

if it still fails check the error logs in github actions pipeline and see what error is it.

[wozniak-slawomir]

Thanks for the response @vicky7230 . I did change it but no result.
I don't have any errors in the pipeline.

"OPENAI_KEY not set" is always displayed - I checked this in repo, so that's not the issue.

I don't seem do get any comments/replies anything at all on any pull request.