GPU requirements

Local LLM compatibility by GPU VRAM

Choose a GPU preset by practical local-fit tier. Each card shows how many models fit cleanly, how many need RAM/offload, and where cloud or larger hardware becomes the better route.

Open interactive calculator

GPU presets10
Model dataset34
Most clean fits33
Best presetApple Silicon 256 GB unified memory
Green

Model fits into clean VRAM/unified memory planning capacity.

Yellow

Possible with RAM/offload or tighter settings, but slower or less comfortable.

Red

Use a smaller model, cloud GPU, hosted API, or a larger local setup.

Entry and mainstream GPUs

Good for small local chat, routing, and 7B/8B Q4 models
2 presets

6 GB VRAM entry GPU

6 GB VRAM 16 GB RAM GTX 1660, RTX 2060 6GB

Small local chat models only

2 clean fits 22 offload 10 too large
Best clean fits: Qwen3 4B Thinking 2507, Gemma 3 4B Open GPU table ->

8 GB VRAM mainstream GPU

8 GB VRAM 32 GB RAM RTX 3060 Ti, RTX 4060

Good for 7B/8B Q4 models

8 clean fits 20 offload 6 too large
Best clean fits: Qwen3 4B Thinking 2507, Gemma 3 4B, Qwen2.5 Coder 7B Open GPU table ->

Local agent GPUs

Useful for 7B/14B coding agents, bots, and routing layers
3 presets

10 GB VRAM older high-end GPU

10 GB VRAM 32 GB RAM RTX 3080 10GB

Strong 7B/8B, tight for 14B

9 clean fits 19 offload 6 too large
Best clean fits: Qwen3 4B Thinking 2507, Gemma 3 4B, Qwen2.5 Coder 7B Open GPU table ->

12 GB VRAM local agent GPU

12 GB VRAM 32 GB RAM RTX 3060 12GB, RTX 4070

Local routing, agents, and model testing

14 clean fits 14 offload 6 too large
Best clean fits: Qwen3 4B Thinking 2507, Gemma 3 4B, Qwen2.5 Coder 7B Open GPU table ->

16 GB VRAM creator GPU

16 GB VRAM 64 GB RAM RTX 4060 Ti 16GB, RTX 4080

Comfortable 14B Q4, some 20B-class models

15 clean fits 18 offload 1 too large
Best clean fits: Qwen3 4B Thinking 2507, Gemma 3 4B, Qwen2.5 Coder 7B Open GPU table ->

Workstation GPUs

Best for larger coding models, long context, and heavier local workflows
3 presets

24 GB VRAM homelab workstation

24 GB VRAM 64 GB RAM RTX 3090, RTX 4090

Heavy local models and homelab inference

26 clean fits 7 offload 1 too large
Best clean fits: Qwen3 4B Thinking 2507, Gemma 3 4B, Qwen2.5 Coder 7B Open GPU table ->

32 GB VRAM Blackwell workstation

32 GB VRAM 64 GB RAM RTX 5090

Ultra high-end consumer workstation for 30B+ models with extra headroom

28 clean fits 5 offload 1 too large
Best clean fits: Qwen3 4B Thinking 2507, Gemma 3 4B, Qwen2.5 Coder 7B Open GPU table ->

48 GB VRAM workstation

48 GB VRAM 128 GB RAM RTX A6000, L40S 48GB

Large local models and long context

29 clean fits 4 offload 1 too large
Best clean fits: Qwen3 4B Thinking 2507, Gemma 3 4B, Qwen2.5 Coder 7B Open GPU table ->

Unified memory systems

Apple Silicon-style shared memory, not directly comparable to discrete VRAM
2 presets

Apple Silicon 32 GB unified memory

32 GB unified 32 GB RAM M2 Max 32GB, M3 Max 36GB

Unified memory; not directly comparable to discrete VRAM

26 clean fits 1 offload 7 too large
Best clean fits: Qwen3 4B Thinking 2507, Gemma 3 4B, Qwen2.5 Coder 7B Open GPU table ->

Apple Silicon 256 GB unified memory

256 GB unified 256 GB RAM Mac Studio M3 Ultra 256GB, Mac Studio M4 Ultra 256GB

Very large unified-memory local AI experiments; still slow for huge MoE models

33 clean fits 0 offload 1 too large
Best clean fits: Qwen3 4B Thinking 2507, Gemma 3 4B, Qwen2.5 Coder 7B Open GPU table ->