You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Please avoid creating issues regarding the following questions, as they might be closed without a response.
Kept up to date by the maintainers. Check here before opening an issue. See also the Roadmap and CONTRIBUTING.md.
Requirements
Q: What hardware and OS are supported?
x86_64 only, with an NVIDIA GPU from Ampere (RTX 30 series) up and driver r580+ (CUDA 13). Windows and Linux through the Desktop app; Linux through pip. macOS, AMD, DGX Spark (aarch64) are on the Roadmap.
Q: How much RAM do I need?
For MoE models the experts live in host RAM, so you need roughly the size of the expert weights in free RAM. For example Qwen/Qwen3.6-35B-A3B (bf16) needs about 70 GB; the NVFP4 checkpoint of the same model needs far less.
Q: Which models are supported?
See models.md. Other checkpoints of the same architectures usually work too. GGUF is currently limited to Gemma-4; support for more GGUF quantization types and architectures is on the Roadmap.
Note
Please avoid creating issues regarding the following questions, as they might be closed without a response.
Kept up to date by the maintainers. Check here before opening an issue. See also the Roadmap and CONTRIBUTING.md.
Requirements
Q: What hardware and OS are supported?
x86_64 only, with an NVIDIA GPU from Ampere (RTX 30 series) up and driver r580+ (CUDA 13). Windows and Linux through the Desktop app; Linux through
pip. macOS, AMD, DGX Spark (aarch64) are on the Roadmap.Q: How much RAM do I need?
For MoE models the experts live in host RAM, so you need roughly the size of the expert weights in free RAM. For example
Qwen/Qwen3.6-35B-A3B(bf16) needs about 70 GB; the NVFP4 checkpoint of the same model needs far less.Q: Which models are supported?
See models.md. Other checkpoints of the same architectures usually work too. GGUF is currently limited to Gemma-4; support for more GGUF quantization types and architectures is on the Roadmap.