Model requirements

Local LLM GPU requirements by model

Pick a model page to see estimated VRAM needs, Q4 fit across common GPUs, Ollama command, and calculator links.

Llama 3.1 8B Instruct

8B · Q4 about 6.00 GB · llama3.1:8b

Llama 3.1 70B Instruct

70B · Q4 about 44 GB · llama3.1:70b

Qwen2.5 Coder 7B

7B · Q4 about 5.50 GB · qwen2.5-coder:7b

Qwen2.5 Coder 14B

14B · Q4 about 10.5 GB · qwen2.5-coder:14b

Qwen2.5 Coder 32B

32B · Q4 about 21 GB · qwen2.5-coder:32b

Qwen3 8B

8B · Q4 about 6.00 GB · qwen3:8b

DeepSeek R1 Distill Qwen 8B

8B · Q4 about 6.00 GB · deepseek-r1:8b

DeepSeek R1 Distill Qwen 14B

14B · Q4 about 10.5 GB · deepseek-r1:14b

DeepSeek R1 Distill Qwen 32B

32B · Q4 about 21 GB · deepseek-r1:32b

Gemma 3 4B

4B · Q4 about 3.50 GB · gemma3:4b

Gemma 3 12B

12B · Q4 about 9.00 GB · gemma3:12b

Gemma 3 27B

27B · Q4 about 18 GB · gemma3:27b

Mistral 7B

7B · Q4 about 5.50 GB · mistral:7b

Mixtral 8x7B

46.7B · Q4 about 28 GB · mixtral:8x7b

Phi-4 14B

14B · Q4 about 10.5 GB · phi4:14b