Nemotron 3 Super recipe for DGX Station - #721
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![gemini-code-assist[bot]](https://cdn.statically.io/img/avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the model configuration for NVIDIA-Nemotron-3-Super-120B-A12B-BF16 by changing the tool-calling parser to qwen3_xml, adding speculative decoding as an opt-in feature, introducing a new dgx_station_gb300 hardware configuration, and updating the prerequisites guide. The feedback highlights a CLI argument syntax error where --mamba_ssm_cache_dtype should use hyphens instead of underscores, and notes that the launch commands in the guide section should be updated to reference qwen3_xml for consistency.
| - "fp8" | ||
| - "--mamba-cache-mode" | ||
| - "align" | ||
| - "--mamba_ssm_cache_dtype" |
There was a problem hiding this comment.
The CLI argument --mamba_ssm_cache_dtype uses an underscore, but vLLM CLI arguments use hyphens. It should be --mamba-ssm-cache-dtype to avoid CLI parsing errors.
- "--mamba-ssm-cache-dtype"| - "--enable-auto-tool-choice" | ||
| - "--tool-call-parser" | ||
| - "qwen3_coder" | ||
| - "qwen3_xml" |
There was a problem hiding this comment.
The default tool-call parser has been updated to qwen3_xml here, but the launch commands in the guide section (lines 147, 159, and 170) still reference qwen3_coder. Please update those commands in the guide to use qwen3_xml as well to ensure consistency.
13ee9a7 to
52be356
Compare
Signed-off-by: Prerit Rodney <240578995+prodney-dev@users.noreply.github.com>
52be356 to
abc7524
Compare
2 participants
No description provided.