GPU compatibility

What LLMs can run on 8 GB VRAM mainstream GPU?

Good for 7B/8B Q4 models Examples: RTX 3060 Ti, RTX 4060, RTX 3070.

Open calculator with this GPU preset

Preset VRAM8.00 GB
System memory32 GB
Use caseGood for 7B/8B Q4 models

Best clean fits

Gemma 3 4B

Small multimodal local assistant and low-resource setups

Qwen2.5 Coder 7B

Small local coding assistant and agent tool generation

Mistral 7B

Fast local chat and simple agent tasks

Llama 3.1 8B Instruct

Fast local chat, lightweight agents, low-cost local testing

Q4 model fit table

ModelSizeQ4 needStatusCalculator
Gemma 3 4B4B3.50 GBRuns locallyOpen calculator
Qwen2.5 Coder 7B7B5.50 GBRuns locallyOpen calculator
Mistral 7B7B5.50 GBRuns locallyOpen calculator
Llama 3.1 8B Instruct8B6.00 GBRuns locallyOpen calculator
Qwen3 8B8B6.00 GBRuns locallyOpen calculator
DeepSeek R1 Distill Qwen 8B8B6.00 GBRuns locallyOpen calculator
Gemma 3 12B12B9.00 GBRAM offloadOpen calculator
Qwen2.5 Coder 14B14B10.5 GBRAM offloadOpen calculator
DeepSeek R1 Distill Qwen 14B14B10.5 GBRAM offloadOpen calculator
Phi-4 14B14B10.5 GBRAM offloadOpen calculator
Gemma 3 27B27B18 GBRAM offloadOpen calculator
Qwen2.5 Coder 32B32B21 GBRAM offloadOpen calculator
DeepSeek R1 Distill Qwen 32B32B21 GBRAM offloadOpen calculator
Mixtral 8x7B46.7B28 GBRAM offloadOpen calculator
Llama 3.1 70B Instruct70B44 GBToo largeOpen calculator