Run LLMs on your own hardware. Find the right launcher, engine, and configuration for your setup.
NVIDIA-first. Mac-strong. Pick your GPU, get your stack.
| Name | Role | Backends | Formats | Score | Install |
|---|---|---|---|---|---|
| CUDA Runtime NVIDIA's parallel computing platform for GPU acceleration | Backend | cuda | - | — | 🐧🪟 |
| ROCm AMD's open-source GPU computing platform | Backend | rocm | - | — | 🐧 |
| Metal Apple's GPU framework for Apple Silicon acceleration | Backend | metal | - | — | |
| Vulkan Cross-platform GPU API for compute and graphics | Backend | vulkan | - | — |