Run LLMs on your own hardware. Find the right launcher, engine, and configuration for your setup.
NVIDIA-first. Mac-strong. Pick your GPU, get your stack.
| Name | Role | Backends | Formats | Score | Install |
|---|---|---|---|---|---|
| Text Generation WebUI Gradio web UI for running Large Language Models | Launcher | cuda, metal, rocm... | gguf, gptq... | 97 (A+) | 🍎🐧🪟 |
| KoboldCpp Easy-to-use AI text generation with llama.cpp backend | Launcher | cuda, metal, rocm... | gguf, ggml | 92 (A+) | 🍎🐧🪟 |
| Ollama Get up and running with large language models locally | Launcher | cuda, metal, rocm... | gguf | 90 (A+) | 🍎🐧🪟 |
| Jan Open-source ChatGPT alternative that runs offline | Launcher | cuda, metal, cpu... | gguf | 87 (A) | 🍎🐧🪟 |
| LM Studio Discover, download, and run local LLMs with a beautiful GUI | Launcher | cuda, metal, cpu... | gguf | 87 (A) | 🍎🐧🪟 |
| LocalAI Free, open-source OpenAI alternative with local inference | Launcher | cuda, metal, rocm... | gguf, safetensors | 77 (B+) | 🍎🐧 |
| GPT4All Free-to-use, locally running, privacy-aware chatbot | Launcher | cuda, metal, cpu | gguf | 72 (B) | 🍎🐧🪟 |
| Open WebUI User-friendly WebUI for LLMs with Ollama/OpenAI support | Launcher | cuda, metal, rocm... | gguf | 62 (C+) | 🍎🐧 |