RedditDiscordGitHub

Fine-tuning LLMs on AMD GPUs with Unsloth Guide

Learn how for fine-tune large language models (LLMs) on AMD GPUs with Unsloth.

You can now fine-tune LLMs on your own local AMD setup with Unsloth. Unsloth supports AMD Radeon RX, MI300X's (192GB) GPUs and more.

1

Make a new isolated environment (Optional)

To not break any system packages, you can make an isolated pip environment. Reminder to check what Python version you have! It might be pip3, pip3.13, python3, python.3.13 etc.

apt install python3.10-venv python3.11-venv python3.12-venv python3.13-venv -y

python -m venv unsloth_env
source unsloth_env/bin/activate
2

Install PyTorch

Install the latest PyTorch, TorchAO, Xformers from https://pytorch.org/ Check your ROCM version via amd-smi version then change https://download.pytorch.org/whl/rocm7.0 to match it.

uv pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm7.0 --upgrade --force-reinstall

We also wrote a single terminal command to extract the correct ROCM version if it helps.

ROCM_TAG="$({ command -v amd-smi >/dev/null 2>&1 && amd-smi version 2>/dev/null | awk -F'ROCm version: ' 'NF>1{split($2,a,"."); print "rocm"a[1]"."a[2]; ok=1; exit} END{exit !ok}'; } || { [ -r /opt/rocm/.info/version ] && awk -F. '{print "rocm"$1"."$2; exit}' /opt/rocm/.info/version; } || { command -v hipconfig >/dev/null 2>&1 && hipconfig --version 2>/dev/null | awk -F': *' '/HIP version/{split($2,a,"."); print "rocm"a[1]"."a[2]; ok=1; exit} END{exit !ok}'; } || { command -v dpkg-query >/dev/null 2>&1 && ver="$(dpkg-query -W -f="${Version}\n" rocm-core 2>/dev/null)" && [ -n "$ver" ] && awk -F'[.-]' '{print "rocm"$1"."$2; exit}' <<<"$ver"; } || { command -v rpm >/dev/null 2>&1 && ver="$(rpm -q --qf '%{VERSION}\n' rocm-core 2>/dev/null)" && [ -n "$ver" ] && awk -F'[.-]' '{print "rocm"$1"."$2; exit}' <<<"$ver"; })"; [ -n "$ROCM_TAG" ] && uv pip install torch torchvision torchaudio --index-url "https://download.pytorch.org/whl/$ROCM_TAG" --upgrade --force-reinstall
3

Install Unsloth

Install Unsloth's dedicated AMD branch:

pip install --no-deps unsloth unsloth-zoo
pip install --no-deps git+https://github.com/unslothai/unsloth-zoo.git
pip install "unsloth[amd] @ git+https://github.com/unslothai/unsloth"
4

Start fine-tuning with Unsloth!

And that's it. Try some examples in our Unsloth Notebooks page!

You can view our dedicated fine-tuning or reinforcement learning guides.

🔢 Reinforcement Learning on AMD GPUs

You can use our 📒gpt-oss RL auto win 2048 example on a MI300X (192GB) GPU. The goal is to play the 2048 game automatically and win it with RL. The LLM (gpt-oss 20b) auto devises a strategy to win the 2048 game, and we calculate a high reward for winning strategies, and low rewards for failing strategies.

The reward over time is increasing after around 300 steps or so!

The goal for RL is to maximize the average reward to win the 2048 game.

We used an AMD MI300X machine (192GB) to run the 2048 RL example with Unsloth, and it worked well!

You can also use our 📒automatic kernel gen RL notebook also with gpt-oss to auto create matrix multiplication kernels in Python. The notebook also devices multiple methods to counteract reward hacking.

The prompt we used to auto create these kernels was:

The RL process learns for example how to apply the Strassen algorithm for faster matrix multiplication inside of Python.

📚AMD Free One-click notebooks

AMD provides one-click notebooks equipped with free 192GB VRAM MI300X GPUs through their Dev Cloud. Train large models completely for free (no signup or credit card required):

Loading GitHub Notebook - AMD Dev Cloudoneclickamd.ai

You can use any Unsloth notebook by prepending https://oneclickamd.ai/github/unslothai/notebooks/blob/main/nb in Unsloth Notebooks by changing the link from https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Gemma3_(270M).ipynb to https://oneclickamd.ai/github/unslothai/notebooks/blob/main/nb/Gemma3_(270M).ipynb

PreviousWindowsNextIntel

Last updated

Was this helpful?