GPU compatibility

What LLMs can run on 8 GB VRAM mainstream GPU?

Good for 7B/8B Q4 models. Examples: RTX 3060 Ti, RTX 4060, RTX 3070.

Open calculator with this GPU preset

Clean local fits8
Offload / slower20
Too large6
Planning capacity6.5 GB clean VRAM

Recommended route: local with limits

This preset is useful locally, but model choice matters. Stay close to the green list for the best experience.

Use locally forsmall chat models, coding helpers, Telegram bots, routing layers, and selected 7B-14B Q4 models.
Watch out foryellow models may run slowly because they need RAM/offload or reduced context settings.
Fallback triggerUse hosted API or cloud GPU when the chosen model lands in yellow for production work.
Green

8 models fit inside the clean planning capacity.

Yellow

20 models can run with RAM/offload tradeoffs.

Red

Examples to avoid locally: Llama 3.1 70B Instruct, Qwen3-Next 80B-A3B Instruct, Qwen3-Coder-Next.

Best clean fits

Start here for responsive local inference.
8 models

Qwen3 4B Thinking 2507

4.02B Runs locally Q4 about 3.2 GB

Small local reasoning, routing, tool decisions, and lightweight coding on 6GB-8GB GPUs

Best local fit Open model page ->

Gemma 3 4B

4B Runs locally Q4 about 3.5 GB

Small multimodal local assistant and low-resource setups

Best local fit Open model page ->

Qwen2.5 Coder 7B

7B Runs locally Q4 about 5.5 GB

Small local coding assistant and agent tool generation

Best local fit Open model page ->

Mistral 7B

7B Runs locally Q4 about 5.5 GB

Fast local chat and simple agent tasks

Best local fit Open model page ->

Possible with offload

Use only when slower generation and tighter settings are acceptable.
20 models

Qwen3.5 9B

9B RAM offload Q4 about 6.6 GB

Modern multimodal local assistant, agent experiments, and coding support on mainstream GPUs

Possible, slower Open model page ->

Gemma 3 12B

12B RAM offload Q4 about 9 GB

Balanced multimodal local chat on 12GB+ GPUs

Possible, slower Open model page ->

Gemma 4 E4B

4B RAM offload Q4 about 9.6 GB

Efficient multimodal local assistant and edge-style agent workflows

Possible, slower Open model page ->

Qwen2.5 Coder 14B

14B RAM offload Q4 about 10.5 GB

Local coding, scripts, repo assistance, technical agents

Possible, slower Open model page ->

Full Q4 model fit table

ModelSizeQ4 needStatusCalculator
Qwen3 4B Thinking 25074.02B3.2 GBRuns locallyOpen calculator
Gemma 3 4B4B3.5 GBRuns locallyOpen calculator
Qwen2.5 Coder 7B7B5.5 GBRuns locallyOpen calculator
Mistral 7B7B5.5 GBRuns locallyOpen calculator
Llama 3.1 8B Instruct8B6 GBRuns locallyOpen calculator
Qwen3 8B8B6 GBRuns locallyOpen calculator
DeepSeek-R1-0528-Qwen3-8B8B6 GBRuns locallyOpen calculator
Qwen3-VL 8B Instruct8B6.5 GBRuns locallyOpen calculator
Qwen3.5 9B9B6.6 GBRAM offloadOpen calculator
Gemma 3 12B12B9 GBRAM offloadOpen calculator
Gemma 4 E4B4B9.6 GBRAM offloadOpen calculator
Qwen2.5 Coder 14B14B10.5 GBRAM offloadOpen calculator
DeepSeek R1 Distill Qwen 14B14B10.5 GBRAM offloadOpen calculator
Phi-4 14B14B10.5 GBRAM offloadOpen calculator
gpt-oss 20B20B14 GBRAM offloadOpen calculator
Devstral Small 2 24B24B15 GBRAM offloadOpen calculator
Qwen3.5 27B27B17 GBRAM offloadOpen calculator
Qwen3.6 27B27B17 GBRAM offloadOpen calculator
Gemma 3 27B27B18 GBRAM offloadOpen calculator
Qwen3 30B-A3B Instruct 250730.5B19 GBRAM offloadOpen calculator
Qwen3-Coder 30B-A3B30B19 GBRAM offloadOpen calculator
GLM-4.7-Flash30B19 GBRAM offloadOpen calculator
Gemma 4 31B31B20 GBRAM offloadOpen calculator
Qwen3-VL 30B-A3B Instruct30B20 GBRAM offloadOpen calculator
Qwen2.5 Coder 32B32B21 GBRAM offloadOpen calculator
DeepSeek R1 Distill Qwen 32B32B21 GBRAM offloadOpen calculator
Qwen3.6 35B-A3B35B24 GBRAM offloadOpen calculator
Mixtral 8x7B46.7B28 GBRAM offloadOpen calculator
Llama 3.1 70B Instruct70B44 GBToo largeOpen calculator
Qwen3-Next 80B-A3B Instruct80B50 GBToo largeOpen calculator
Qwen3-Coder-Next80B52 GBToo largeOpen calculator
gpt-oss 120B120B65 GBToo largeOpen calculator
Devstral 2 123B123B75 GBToo largeOpen calculator
GLM-5.2744B466 GBToo largeOpen calculator