Intel hardware is the quietly practical option for local AI on Linux: Arc discrete GPUs are inexpensive per gigabyte of VRAM, integrated Xe graphics are already in machines you own, and Core Ultra laptops ship an NPU built for sustained low-power inference. The kernel side is fully open and in-tree, so there is no proprietary module to break on updates.
The stack is unfamiliar rather than difficult. This covers the driver layers, OpenVINO, choosing a device, and running LLMs through SYCL or Vulkan.
The Stack, Layer by Layer
| Layer | Component | Where it comes from |
|---|---|---|
| Kernel driver | i915 or xe | Mainline kernel — already present |
| Compute runtime | intel-opencl-icd, Level Zero | Distro or Intel repository packages |
| Framework | OpenVINO, PyTorch XPU, SYCL | pip or Intel packages |
| NPU driver | intel-driver-compiler-npu + level-zero-npu | Intel packages; recent kernel required |
i915 vs xe — the newer xe driver is taking over for recent Intel graphics while i915 covers older generations. Which one binds your card depends on your kernel version — both are in-tree.
Level Zero is the important one — OpenVINO and SYCL reach the GPU and NPU through Level Zero, not OpenCL. If Level Zero is missing, your device simply will not appear.
Installing the Compute Runtime
# Ubuntu — Intel compute packages
$ sudo apt install -y intel-opencl-icd intel-level-zero-gpu level-zero clinfo
# Confirm the GPU is visible to the compute stack
$ clinfo | grep -i "device name"
# Which kernel driver bound the card?
$ lspci -k | grep -A3 -i vga
As with AMD ROCm, device access is group-controlled. Your user must be in the render group or every compute call fails with a permission error:
$ sudo usermod -aG render,video $USER
# Log out and back in — group changes need a new session
$ groups
$ ls -l /dev/dri/
Installing OpenVINO
OpenVINO is Intel's inference runtime. It installs as a Python package and does the useful thing of abstracting the device, so the same code targets CPU, GPU, or NPU:
$ python3 -m venv .venv && source .venv/bin/activate
$ pip install openvino
# What devices did it find?
$ python -c "import openvino as ov; print(ov.Core().available_devices)"
# Expect entries like ['CPU', 'GPU', 'NPU']
That one command is the real test of your setup. If GPU or NPU is missing, the runtime layer is incomplete — go back to the compute runtime step rather than debugging your model code.
Choosing a Device
import openvino as ov
core = ov.Core()
print(core.available_devices)
# Compile for a specific device
model = core.read_model("model.xml")
compiled = core.compile_model(model, "GPU") # or "CPU", "NPU"
# Let OpenVINO choose, with a preference order
compiled = core.compile_model(model, "AUTO:GPU,CPU")
| Device | Strength | Use for |
|---|---|---|
| CPU | Always available, flexible | Fallback; small models; development |
| GPU | Highest throughput | Batch work, image models, larger LLMs |
| NPU | Efficiency per watt | Always-on tasks on battery; sustained light inference |
NPU is about power, not peak speed — an NPU typically will not beat a discrete GPU on raw throughput. Its value is running continuous inference at very low power — wake words, background classification, always-on features on a laptop.
AUTO is a reasonable default — it picks a device and falls back gracefully, which matters when the same code ships to machines with different hardware.
Running LLMs on Intel Hardware
Two practical routes, and the second is easier than the first:
SYCL backend in llama.cpp — the native path for Intel GPUs, built against oneAPI:
# With oneAPI installed and its environment sourced
$ source /opt/intel/oneapi/setvars.sh
$ cmake -B build -DGGML_SYCL=ON -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx
$ cmake --build build --config Release -j $(nproc)
$ ./build/bin/llama-cli -m models/model.gguf -ngl 99
Vulkan backend — vendor-neutral, and by far the quickest to get working on Intel:
$ sudo apt install -y libvulkan-dev glslc vulkan-tools mesa-vulkan-drivers
$ vulkaninfo --summary # confirm the Intel GPU appears
$ cmake -B build -DGGML_VULKAN=ON
$ cmake --build build --config Release -j $(nproc)
If you want GPU inference on Intel today with minimum effort, use Vulkan. Reach for SYCL when you need the extra performance or are already invested in oneAPI. Full build details are in our llama.cpp guide.
Enabling the NPU
NPU support is the newest part of the stack and the most version-sensitive. It needs a recent kernel with the intel_vpu driver plus Intel's NPU userspace packages:
# Is the NPU present and bound?
$ ls /dev/accel/
$ lsmod | grep intel_vpu
$ dmesg | grep -i vpu
# After installing Intel's NPU driver packages:
$ python -c "import openvino as ov; print(ov.Core().available_devices)"
Kernel version is decisive — NPU support arrived relatively recently. On an older LTS kernel the device will not appear at all — check your kernel before debugging anything else.
Not every model runs on NPU — NPUs support a narrower set of operations than GPUs. OpenVINO will refuse to compile unsupported models for NPU; fall back to GPU or CPU for those.
Group access again — the /dev/accel nodes are group-owned like the DRI nodes. Same render group requirement.
Troubleshooting
available_devices shows only CPU — the compute runtime or Level Zero package is missing, or your user is not in the render group. Check clinfo first — if it shows no device, the problem is below OpenVINO.
Permission denied on /dev/dri/renderD* — add your user (or service account) to render and video, then start a new login session.
clinfo works but OpenVINO does not see the GPU — you have OpenCL but not Level Zero. Install the level-zero and intel-level-zero-gpu packages.
NPU never appears — usually an older kernel without the intel_vpu driver, or missing NPU userspace packages. Verify /dev/accel exists.
Model fails to compile for NPU — an unsupported operation. Compile that model for GPU or CPU instead — this is expected, not a misconfiguration.
Poor performance on integrated graphics — integrated GPUs share system RAM, so bandwidth is the bottleneck. Small models only, and expect modest gains over CPU.
Frequently Asked Questions
Can Intel Arc GPUs run AI workloads on Linux?
Yes. The kernel driver is in-tree, and with Intel's compute runtime plus Level Zero installed you can run OpenVINO, PyTorch XPU, or llama.cpp through the SYCL or Vulkan backends. Arc cards are attractive for local inference because of their VRAM per unit cost.
What is the difference between an Intel GPU and NPU for AI?
The GPU delivers higher throughput and suits batch work and larger models. The NPU is optimized for efficiency per watt, making it right for always-on, low-power inference such as wake-word detection on a laptop. NPUs also support a narrower set of operations.
Do I need OpenVINO to use Intel hardware for AI?
No, but it is the most direct path, and it is the only one of these that targets the NPU. For running GGUF language models, llama.cpp with the Vulkan or SYCL backend works without OpenVINO.
Why does OpenVINO only list CPU as an available device?
The Level Zero runtime is missing, or your user is not in the render group. Run clinfo to check whether the compute stack sees the GPU at all — if it does not, the problem is in the driver layer rather than in OpenVINO.
Is SYCL or Vulkan better for Intel GPUs in llama.cpp?
SYCL is the native path and generally performs better, but requires oneAPI. Vulkan needs only Mesa and Vulkan headers and works in minutes, which makes it the pragmatic first choice — switch to SYCL if you need the extra performance.