📚

Recommended Stacks

Pre-configured combinations of launcher + engine + format + quantization for your hardware.

🟢 NVIDIA 🍎 Mac (Apple Silicon)💻 CPU Only 🔴 AMD

🔴 AMD Stacks

AMD GPU users - ROCm ecosystem

16GB VRAM (RX 7900 XT)

16GB VRAM

Beginner

Stack

ollama + llama.cpp

Formats

gguf

Quantization

Q4_K_M to Q5_K_M

💡 ROCm supported. Setup more complex than NVIDIA

Power

Stack

text-generation-webui + llama.cpp

Formats

gguf

Quantization

Q5_K_M

💡 ROCm environment setup required

24GB VRAM (RX 7900 XTX)

24GB VRAM

Beginner

Stack

ollama + llama.cpp

Formats

gguf

Quantization

Q5_K_M to Q6_K

💡 70B Q4 within reach

Power

Stack

vllm + vllm

Formats

safetensors

Quantization

FP16

💡 vLLM ROCm compatible