Skip to content

Gate Reasoning(effort=...) on registry support#528

Merged
0xallam merged 1 commit into
mainfrom
fix/gpt4o-reasoning
Jun 8, 2026
Merged

Gate Reasoning(effort=...) on registry support#528
0xallam merged 1 commit into
mainfrom
fix/gpt4o-reasoning

Conversation

@0xallam

@0xallam 0xallam commented Jun 8, 2026

Copy link
Copy Markdown
Member

OpenAI's Responses API rejects reasoning.effort on non-reasoning models (gpt-4o, etc.) with unsupported_parameter. Scans with the default STRIX_REASONING_EFFORT=high crashed at the first model call.

Lifts model_supports_reasoning to a public helper (handles litellm//any-llm//openai/ prefixes + falls back to last-segment lookup), gates Reasoning(effort=...) on it in make_model_settings, and unifies uses_chat_completions_tool_schema to share the same helper.

22/22 cases verified across gpt-4o, gpt-5.x, o-series, claude-4.x, deepseek, gemini, groq, mistral, xai, openrouter, plus legacy litellm/... prefix form and unknown SKUs.

OpenAI's Responses API rejects reasoning.effort on non-reasoning
models like gpt-4o with `unsupported_parameter`, so any scan with
the default STRIX_REASONING_EFFORT=high against gpt-4o crashed at
the first model call. drop_params=True absorbs the rejected param
on LiteLLM-routed models but the SDK's native OpenAI path has no
equivalent.

Lift model_supports_reasoning to a public helper that strips
litellm/, any-llm/, openai/ prefixes and falls back to last-segment
lookup so prefixed forms like anthropic/claude-opus-4-7 resolve
through the bare model_cost entry. make_model_settings regains
model_name and skips Reasoning() when the registry doesn't confirm
support. uses_chat_completions_tool_schema reuses the same helper
(was duplicating the lookup under a misleading name).
@0xallam 0xallam merged commit 04eb03f into main Jun 8, 2026
@0xallam 0xallam deleted the fix/gpt4o-reasoning branch June 8, 2026 20:18
@greptile-apps

greptile-apps Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes crashes caused by STRIX_REASONING_EFFORT being forwarded to non-reasoning models (e.g., gpt-4o) when using OpenAI's Responses API, which rejects reasoning.effort on unsupported models. The fix gates Reasoning(effort=...) on a new public helper model_supports_reasoning, which handles provider-prefixed model names and falls back to the last path segment for unknown prefixes.

  • model_supports_reasoning replaces the private _supports_responses_custom_tools, extending prefix stripping to litellm/, any-llm/, and openai/, and adding a last-segment fallback for unrecognised provider paths.
  • make_model_settings now requires a model_name keyword argument and only applies Reasoning(effort=...) when the model is confirmed as a reasoning model; uses_chat_completions_tool_schema is updated to call the same helper.

Confidence Score: 4/5

Safe to merge; the crash fix is correct and the only call site for make_model_settings has been updated.

The change correctly prevents reasoning effort from being sent to non-reasoning models via the Responses API. The renamed helper now drives both the new reasoning gate and the existing tool-schema routing; the shared last-segment fallback logic is new and could affect tool schema routing for openai// style names.

strix/config/models.py — the renamed helper drives both the reasoning gate and tool-schema routing; the new fallback logic is worth a second look.

Important Files Changed

Filename Overview
strix/config/models.py Renames private _supports_responses_custom_tools to public model_supports_reasoning, extends prefix stripping, and adds a last-segment fallback; uses_chat_completions_tool_schema updated to call the new helper with a subtle behavioral change for openai/<path>/<model> names
strix/core/inputs.py Adds model_name keyword-only parameter to make_model_settings and gates Reasoning(effort=...) on model_supports_reasoning; straightforward and correct fix
strix/core/runner.py Passes model_name=resolved_model to the updated make_model_settings; the only call site, correctly updated

Comments Outside Diff (1)

  1. strix/config/models.py, line 115-122 (link)

    P2 uses_chat_completions_tool_schema now silently couples two independent decisions

    After this change, routing to the Responses API tool schema is decided entirely by model_supports_reasoning. This works today because every OpenAI model that uses the Responses API is a reasoning model, but the invariant isn't enforced anywhere. A future model that supports the Responses API without supports_reasoning=True in litellm's model cost table would be incorrectly sent through the chat-completions schema — potentially producing silently wrong tool calls without any error at runtime.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: strix/config/models.py
    Line: 115-122
    
    Comment:
    **`uses_chat_completions_tool_schema` now silently couples two independent decisions**
    
    After this change, routing to the Responses API tool schema is decided entirely by `model_supports_reasoning`. This works today because every OpenAI model that uses the Responses API is a reasoning model, but the invariant isn't enforced anywhere. A future model that supports the Responses API without `supports_reasoning=True` in litellm's model cost table would be incorrectly sent through the chat-completions schema — potentially producing silently wrong tool calls without any error at runtime.
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
strix/config/models.py:115-122
**`uses_chat_completions_tool_schema` now silently couples two independent decisions**

After this change, routing to the Responses API tool schema is decided entirely by `model_supports_reasoning`. This works today because every OpenAI model that uses the Responses API is a reasoning model, but the invariant isn't enforced anywhere. A future model that supports the Responses API without `supports_reasoning=True` in litellm's model cost table would be incorrectly sent through the chat-completions schema — potentially producing silently wrong tool calls without any error at runtime.

### Issue 2 of 2
strix/config/models.py:133-136
**Last-segment fallback can produce false positives for openrouter-style deep paths**

`name.rsplit("/", 1)[1]` resolves `openrouter/openai/o3` to `o3` — which is fine for the `make_model_settings` path because those models travel through litellm's `drop_params=True` layer and a wrong flag is a no-op. However, the same fallback is reached from `uses_chat_completions_tool_schema` for any `openai/<provider>/<model>` pattern: if the provider segment is non-empty after stripping `openai/` and the resulting name still contains `/`, the fallback kicks in and the final model segment drives the Responses-vs-chat-completions routing decision. Explicit testing with a path like `openai/custom/o3` would confirm whether this edge case is intentional.

Reviews (1): Last reviewed commit: "Gate Reasoning(effort=...) on registry s..." | Re-trigger Greptile

Comment thread strix/config/models.py
Comment on lines +133 to +136
entry = litellm.model_cost.get(name)
if entry is None and "/" in name:
entry = litellm.model_cost.get(name.rsplit("/", 1)[1])
return bool(entry and entry.get("supports_reasoning"))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Last-segment fallback can produce false positives for openrouter-style deep paths

name.rsplit("/", 1)[1] resolves openrouter/openai/o3 to o3 — which is fine for the make_model_settings path because those models travel through litellm's drop_params=True layer and a wrong flag is a no-op. However, the same fallback is reached from uses_chat_completions_tool_schema for any openai/<provider>/<model> pattern: if the provider segment is non-empty after stripping openai/ and the resulting name still contains /, the fallback kicks in and the final model segment drives the Responses-vs-chat-completions routing decision. Explicit testing with a path like openai/custom/o3 would confirm whether this edge case is intentional.

Prompt To Fix With AI
This is a comment left during a code review.
Path: strix/config/models.py
Line: 133-136

Comment:
**Last-segment fallback can produce false positives for openrouter-style deep paths**

`name.rsplit("/", 1)[1]` resolves `openrouter/openai/o3` to `o3` — which is fine for the `make_model_settings` path because those models travel through litellm's `drop_params=True` layer and a wrong flag is a no-op. However, the same fallback is reached from `uses_chat_completions_tool_schema` for any `openai/<provider>/<model>` pattern: if the provider segment is non-empty after stripping `openai/` and the resulting name still contains `/`, the fallback kicks in and the final model segment drives the Responses-vs-chat-completions routing decision. Explicit testing with a path like `openai/custom/o3` would confirm whether this edge case is intentional.

How can I resolve this? If you propose a fix, please make it concise.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant