GPU compatibility

What LLMs can run on 6 GB VRAM entry GPU?

Small local chat models only Examples: GTX 1660, RTX 2060 6GB.

Open calculator with this GPU preset

Preset VRAM6.00 GB
System memory16 GB
Use caseSmall local chat models only

Best clean fits

Gemma 3 4B

Small multimodal local assistant and low-resource setups

Q4 model fit table

ModelSizeQ4 needStatusCalculator
Gemma 3 4B4B3.50 GBRuns locallyOpen calculator
Qwen2.5 Coder 7B7B5.50 GBRAM offloadOpen calculator
Mistral 7B7B5.50 GBRAM offloadOpen calculator
Llama 3.1 8B Instruct8B6.00 GBRAM offloadOpen calculator
Qwen3 8B8B6.00 GBRAM offloadOpen calculator
DeepSeek R1 Distill Qwen 8B8B6.00 GBRAM offloadOpen calculator
Gemma 3 12B12B9.00 GBRAM offloadOpen calculator
Qwen2.5 Coder 14B14B10.5 GBRAM offloadOpen calculator
DeepSeek R1 Distill Qwen 14B14B10.5 GBRAM offloadOpen calculator
Phi-4 14B14B10.5 GBRAM offloadOpen calculator
Gemma 3 27B27B18 GBRAM offloadOpen calculator
Qwen2.5 Coder 32B32B21 GBToo largeOpen calculator
DeepSeek R1 Distill Qwen 32B32B21 GBToo largeOpen calculator
Mixtral 8x7B46.7B28 GBToo largeOpen calculator
Llama 3.1 70B Instruct70B44 GBToo largeOpen calculator