AMD GPUs are capable AI hardware on Linux, and the kernel side is genuinely good — the amdgpu driver is in the mainline kernel, so there is no proprietary module to break on updates. The friction is entirely in ROCm, the userspace compute stack, whose official support list is much narrower than the range of cards that actually work.
This guide gets ROCm installed, verified, and running PyTorch and llama.cpp — including the workaround that makes many "unsupported" consumer Radeon cards work anyway.
How the AMD Stack Fits Together
| Layer | What it is | Where it comes from |
|---|---|---|
amdgpu | Kernel driver — talks to the card | Mainline kernel (already installed) |
| ROCm runtime | HIP, HSA, compute libraries | AMD packages you install |
/dev/kfd | Compute device node — the GPU compute interface | Created by amdgpu at boot |
| Framework | PyTorch ROCm build, llama.cpp HIP backend | Built against ROCm |
Two consequences worth internalizing: your GPU may show up fine for graphics while compute is completely broken (that is /dev/kfd permissions), and ROCm version compatibility is with your framework build, not with the kernel.
Check Support Before You Install
ROCm officially supports a specific set of distros — in practice Ubuntu LTS and the RHEL family — and a specific list of GPUs, weighted heavily toward CDNA data-center cards and higher-end RDNA consumer cards. Find your GPU's target architecture first:
# Identify the card
$ lspci -nn | grep -i vga
# After ROCm is installed, this prints the gfx target (e.g. gfx1100)
$ rocminfo | grep -m1 gfx
That gfxNNNN string is the thing that matters. Check AMD's current compatibility matrix for your card and ROCm release before assuming anything — the list changes between releases, in both directions.
Installing ROCm
AMD ships an installer script that adds the repositories and pulls the right packages. On Ubuntu:
# Download the amdgpu-install package for your distro release
# from AMD's ROCm documentation site, then:
$ sudo apt install ./amdgpu-install_<version>_all.deb
$ sudo apt update
# Install the compute stack (no graphics/display components)
$ sudo amdgpu-install --usecase=rocm --no-dkms
$ sudo reboot
--no-dkms — on a modern kernel you already have amdgpu in-tree. Skipping the DKMS module avoids building an out-of-tree driver you do not need — and avoids one more thing that can break on kernel updates.
--usecase=rocm — installs the compute stack only. Running the installer without a usecase pulls in graphics packages that can conflict with your desktop setup.
Fix Device Permissions — The Step Everyone Skips
Compute access goes through /dev/kfd and /dev/dri/*, which are group-owned. Your user must be in the render and video groups or every compute call fails with a permission error that rarely mentions permissions:
$ sudo usermod -aG render,video $USER
# Group membership only applies to NEW sessions — log out and back in
$ groups
$ ls -l /dev/kfd /dev/dri/renderD*
For a service account (an Ollama or inference daemon running as its own user), add that user to the same groups — this is a very common cause of "works when I run it, fails as a service".
Verifying the Installation
# Full device report — the definitive check
$ rocminfo
# GPU status, temperature, memory, utilization
$ rocm-smi
# Watch during a workload
$ watch -n1 rocm-smi
If rocminfo lists your GPU as an Agent with a gfx name, the stack is working. If it shows only your CPU agent, the GPU is not being exposed — check permissions and whether the card is supported.
PyTorch on ROCm
PyTorch ships ROCm builds through its own wheel index. Critically, the API stays CUDA-shaped — you still call torch.cuda and use device="cuda". Existing code usually runs unmodified:
$ python3 -m venv .venv && source .venv/bin/activate
# Install the ROCm build (match the rocm version in the URL to yours)
$ pip install torch --index-url https://download.pytorch.org/whl/rocm6.2
$ python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"
Do not pip install torch plainly — the default index gives you the CUDA or CPU build. On AMD that silently lands you on CPU, which people then mistake for ROCm being slow.
Making Unsupported Cards Work
Many consumer Radeon cards are not on the official list but work fine, because they are architecturally close to a card that is. HSA_OVERRIDE_GFX_VERSION tells the runtime to treat your GPU as a supported target:
# Present an RDNA3 card as gfx1100
$ export HSA_OVERRIDE_GFX_VERSION=11.0.0
# Present an RDNA2 card as gfx1030
$ export HSA_OVERRIDE_GFX_VERSION=10.3.0
$ python -c "import torch; print(torch.cuda.is_available())"
Set the value matching your card's generation, not an arbitrary one. This is unofficial and unsupported by AMD: it can work perfectly, produce subtly wrong numerics, or crash. Test your actual workload before relying on it, and make it permanent through a systemd Environment= line rather than a shell profile if a service needs it.
llama.cpp and Ollama on ROCm
For local LLM inference, both paths work. llama.cpp builds against HIP directly:
$ HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \
cmake -B build -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1100 -DCMAKE_BUILD_TYPE=Release
$ cmake --build build --config Release -j $(nproc)
$ ./build/bin/llama-cli -m models/model.gguf -ngl 99
Ollama detects ROCm automatically and falls back to CPU when it cannot use the GPU — check ollama ps and the service logs to see which one you actually got. If ROCm is more trouble than it is worth, the Vulkan backend gives you GPU inference with none of this setup.
Troubleshooting
rocminfo lists only the CPU agent — the GPU is not exposed to the compute stack. Check render/video group membership, that /dev/kfd exists, and that the card is supported.
"Permission denied" on /dev/kfd — group membership missing, or you did not start a new login session after usermod. Run groups to confirm.
"HIP error: no ROCm-capable device is detected" — usually an unsupported gfx target. Try HSA_OVERRIDE_GFX_VERSION for your generation.
torch.cuda.is_available() is False — you probably installed the default (CUDA/CPU) PyTorch wheel. Reinstall from the ROCm index URL.
Works interactively, fails as a service — the service user is not in render/video, or the environment override is missing from the unit file.
Crashes or garbage output after an override — the override is not safe for your card. Drop back to CPU or Vulkan for that workload.
Breaks after a kernel upgrade — if you installed the DKMS module, it may have failed to rebuild. On in-tree amdgpu, reinstall the ROCm userspace instead.
Best Practices
1. Verify with rocminfo before installing frameworks — no point debugging PyTorch when the runtime cannot see the card.
2. Match ROCm and framework versions — the PyTorch wheel index encodes a ROCm version — keep them aligned.
3. Prefer in-tree amdgpu — use --no-dkms on modern kernels; fewer out-of-tree modules means fewer update failures.
4. Put service users in render and video — do this when you create the account, not after the first mysterious failure.
5. Treat gfx overrides as experimental — validate output correctness on a known workload before trusting results.
6. Keep Vulkan as a fallback — for inference it is far less setup, and it works across vendors.
Frequently Asked Questions
Do AMD GPUs work for AI on Linux?
Yes. The amdgpu kernel driver is in the mainline kernel and ROCm provides the compute stack, with PyTorch shipping official ROCm builds. The main friction is that ROCm's officially supported GPU and distro list is narrower than the hardware that works in practice.
Why does PyTorch still use torch.cuda on AMD?
The ROCm build of PyTorch deliberately keeps the CUDA-shaped API so existing code runs unmodified. You still write device="cuda" and call torch.cuda functions; they are routed to HIP underneath.
What does HSA_OVERRIDE_GFX_VERSION do?
It makes the ROCm runtime treat your GPU as a different, supported gfx target — commonly 11.0.0 for RDNA3 or 10.3.0 for RDNA2 cards. It often makes unsupported consumer Radeon cards work, but it is unofficial and should be validated on your workload.
Why do I get permission denied on /dev/kfd?
Your user is not in the render and video groups, which own the compute device nodes. Run sudo usermod -aG render,video $USER and start a new login session, since group changes do not apply to existing sessions.
Is Vulkan a real alternative to ROCm?
For LLM inference, yes. llama.cpp's Vulkan backend needs only a working driver plus Vulkan headers and runs on AMD, Intel, and NVIDIA hardware. It does not replace ROCm for PyTorch training, but it avoids the entire ROCm setup for running models.