Local image generation on Linux is mature: two well-established interfaces, both plain Python apps you run in a venv and reach over HTTP. Nothing leaves your machine, there is no per-image cost, and the whole thing runs on hardware you already own.
This guide sets up ComfyUI and AUTOMATIC1111 on Linux, covers realistic VRAM expectations, model file safety, running them as services, and the errors you will hit on the first afternoon.
Which Interface?
| ComfyUI | AUTOMATIC1111 | |
|---|---|---|
| Interface | Node graph — explicit pipeline | Form-based web UI |
| Learning curve | Steeper; you build the workflow | Immediate; fill in fields |
| Reproducibility | Workflow saved in the image metadata | Parameters in the UI |
| Low VRAM behaviour | Generally efficient by design | Needs explicit low-VRAM flags |
| Best for | Complex, repeatable pipelines | Quick generation and experimentation |
If you are unsure, install ComfyUI. It costs an hour of learning and gives you pipelines you can save, share, and run headlessly through its API afterwards.
Hardware Expectations
Image models are much smaller than LLMs, but resolution and batch size drive VRAM hard. Rough guidance:
| VRAM | What is comfortable |
|---|---|
| 4 GB | Small models at modest resolution, with low-VRAM flags; slow |
| 8 GB | Standard SD-family models at typical resolutions |
| 12–16 GB | Larger models, higher resolution, upscaling, ControlNet |
| 24 GB+ | The newest large image models and heavy batching |
CPU-only generation technically works and is genuinely painful — minutes per image rather than seconds. Treat it as a capability check, not a workflow. Plan disk space too: checkpoints run several gigabytes each and collections grow fast.
Prerequisites
You need a working GPU driver, Python with venv, and git. You do not need the CUDA toolkit — PyTorch wheels bundle their own runtime (see our driver and CUDA guide):
# Debian / Ubuntu
$ sudo apt install -y python3-venv python3-pip git
# Verify the GPU is visible
$ nvidia-smi # NVIDIA
$ rocm-smi # AMD
Installing ComfyUI
$ git clone https://github.com/comfyanonymous/ComfyUI
$ cd ComfyUI
$ python3 -m venv .venv && source .venv/bin/activate
# NVIDIA — install PyTorch from the CUDA wheel index
$ pip install torch torchvision --index-url https://download.pytorch.org/whl/cu124
# AMD — use the ROCm wheel index instead
# pip install torch torchvision --index-url https://download.pytorch.org/whl/rocm6.2
$ pip install -r requirements.txt
$ python main.py
ComfyUI serves on http://127.0.0.1:8188. Place checkpoint files in models/checkpoints/, LoRAs in models/loras/, and VAEs in models/vae/, then use the refresh control in the UI to pick them up without a restart.
ComfyUI-Manager — install it early. It handles custom nodes and their dependencies, which is otherwise the most tedious part of running ComfyUI.
Useful flags — --lowvram and --novram for tight cards, --cpu to force CPU, --listen to bind beyond localhost.
Installing AUTOMATIC1111
$ git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui
$ cd stable-diffusion-webui
# webui.sh creates its own venv and installs dependencies on first run
$ ./webui.sh
It serves on http://127.0.0.1:7860, and checkpoints live in models/Stable-diffusion/. Configure launch flags once in webui-user.sh rather than typing them each time:
# webui-user.sh
export COMMANDLINE_ARGS="--xformers --medvram"
# Common flags:
# --medvram / --lowvram reduce VRAM use, slower
# --xformers memory-efficient attention (NVIDIA)
# --no-half-vae fixes black images on some GPUs
# --listen bind to all interfaces (read the warning below)
# --api enable the HTTP API
Model Files: Use safetensors
Checkpoints come in two formats and the distinction is a real security boundary. .ckpt files are Python pickles — loading one executes code. .safetensors files contain only tensor data and cannot execute anything.
Rule — download .safetensors exclusively. There is essentially never a reason to accept a .ckpt from the internet today.
Verify what you downloaded — check file sizes and, where publishers provide them, hashes — a truncated multi-gigabyte download fails in confusing ways.
Storage layout — keep models on a data disk and symlink them in. Both tools accept symlinked model directories, and it keeps your root filesystem from filling.
Running as a systemd Service
Once it is more than a toy, run it as a service so it survives logout and restarts on failure:
# /etc/systemd/system/comfyui.service
[Unit]
Description=ComfyUI
After=network.target
[Service]
Type=simple
User=aiuser
WorkingDirectory=/opt/ComfyUI
ExecStart=/opt/ComfyUI/.venv/bin/python main.py --port 8188
Restart=on-failure
RestartSec=10
[Install]
WantedBy=multi-user.target
$ sudo systemctl daemon-reload
$ sudo systemctl enable --now comfyui
$ journalctl -u comfyui -f
For AMD, add any required environment to the unit — for example Environment="HSA_OVERRIDE_GFX_VERSION=11.0.0" — and make sure the service user is in the render and video groups, as covered in our ROCm guide.
Exposing It Safely
Neither tool has authentication that you should rely on, and both can read and write files on the host. Binding one to 0.0.0.0 on an untrusted network is a genuinely bad idea.
Keep the bind local — leave the app on 127.0.0.1 and reach it through a reverse proxy or SSH tunnel.
SSH tunnel is the zero-config option — ssh -L 8188:localhost:8188 user@host gives you the UI locally with no exposure at all.
Reverse proxy for shared access — terminate TLS and require authentication in nginx — see our nginx config generator.
Never expose to the internet directly — custom nodes and extensions run arbitrary code by design; treat the whole app as trusted-users-only.
Troubleshooting
"CUDA out of memory" — lower the resolution or batch size first, then add --medvram/--lowvram. Restarting frees VRAM held by a previous model.
torch.cuda.is_available() is False — you installed the CPU-only PyTorch wheel. Reinstall from the correct CUDA or ROCm index URL.
Black or corrupted images — a VAE precision problem on some GPUs — try --no-half-vae.
Model does not appear in the UI — it is in the wrong directory, or you need to refresh the model list. Confirm the path matches the tool you are running.
Custom node breaks ComfyUI on startup — read the traceback for the node name, remove that directory from custom_nodes/, and restart. Install nodes one at a time.
Very slow generation — confirm it is actually using the GPU — watch nvidia-smi or rocm-smi during a run; silent CPU fallback is the usual cause.
Disk full — checkpoints, LoRAs, and outputs accumulate quickly. Audit with our disk usage tool and move models to a dedicated volume.
Frequently Asked Questions
How much VRAM do I need for Stable Diffusion?
8 GB is comfortable for standard SD-family models at typical resolutions. 4 GB works with low-VRAM flags at reduced resolution, while 12–16 GB or more is better for larger models, high resolution, upscaling, and ControlNet workflows.
ComfyUI or AUTOMATIC1111 — which should I install?
AUTOMATIC1111 is faster to start with because it is a simple form-based UI. ComfyUI takes longer to learn but represents the pipeline as an explicit node graph, saves workflows inside generated images, and is better for repeatable or automated generation.
Do I need the CUDA toolkit for Stable Diffusion?
No. PyTorch wheels bundle their own CUDA runtime, so a working NVIDIA driver is enough. Install PyTorch from the correct wheel index for your hardware — the CUDA index for NVIDIA, the ROCm index for AMD.
Are .ckpt model files safe to download?
No. The .ckpt format is a Python pickle and loading one can execute arbitrary code. Download .safetensors files only — they contain tensor data with no execution capability, and virtually every model is published in that format now.
Can I run Stable Diffusion without a GPU?
Technically yes, using the CPU flag, but generation takes minutes per image instead of seconds. It is fine for verifying an installation and impractical as a real workflow.