Local LLM model fit
Qwen3.6 35B-A3B is a 35B Qwen3.6 model. This page estimates Q4 VRAM fit, Ollama command, context planning, and fallback choices for common local AI GPUs.
Agentic coding and multimodal reasoning when 27B is not enough and 32GB-class headroom is available. Weakness: The 24GB Ollama Q4 size leaves very little room on single 24GB GPUs once context and runtime overhead are included.
| Hardware | Examples | Clean capacity | Q4 need | Status | Calculator |
|---|---|---|---|---|---|
| 6 GB VRAM entry GPU | GTX 1660, RTX 2060 6GB | 4.5 GB clean VRAM | 24 GB | Too large | Open calculator |
| 8 GB VRAM mainstream GPU | RTX 3060 Ti, RTX 4060, RTX 3070 | 6.5 GB clean VRAM | 24 GB | RAM offload | Open calculator |
| 10 GB VRAM older high-end GPU | RTX 3080 10GB | 8.5 GB clean VRAM | 24 GB | RAM offload | Open calculator |
| 12 GB VRAM local agent GPU | RTX 3060 12GB, RTX 4070, RTX 5070 | 10.5 GB clean VRAM | 24 GB | RAM offload | Open calculator |
| 16 GB VRAM creator GPU | RTX 4060 Ti 16GB, RTX 4080, RTX 5070 Ti, RTX 5080 | 14.5 GB clean VRAM | 24 GB | RAM offload | Open calculator |
| 24 GB VRAM homelab workstation | RTX 3090, RTX 4090 | 22.5 GB clean VRAM | 24 GB | RAM offload | Open calculator |
| 32 GB VRAM Blackwell workstation | RTX 5090 | 30.5 GB clean VRAM | 24 GB | Runs locally | Open calculator |
| 48 GB VRAM workstation | RTX A6000, L40S 48GB | 46.5 GB clean VRAM | 24 GB | Runs locally | Open calculator |
| Apple Silicon 32 GB unified memory | M2 Max 32GB, M3 Max 36GB | 26 GB unified | 24 GB | RAM offload | Open calculator |
| Apple Silicon 256 GB unified memory | Mac Studio M3 Ultra 256GB, Mac Studio M4 Ultra 256GB | 250 GB unified | 24 GB | Runs locally | Open calculator |
| Quantization | Estimated memory | Use case |
|---|---|---|
| Q4 / 4-bit | 24 GB | Default local inference balance |
| Q5 / 5-bit | 30 GB | Better quality, more VRAM |
| Q8 / 8-bit | 48 GB | High quality, much more VRAM |
| FP16 / 16-bit | 96 GB | Mostly workstation/server use |
This is a practical planning estimate, not a benchmark. Real memory use changes with backend, context length, KV cache, quantization file, drivers, and offloading settings.
32B · Strong local coding and architecture work on 24GB GPUs
32B · Heavy local reasoning on 24GB GPUs
31B · High-quality multimodal reasoning, coding assistants, and local-first agent workflows
30.5B · General local reasoning, tool agents, multilingual writing, and coding on 24GB+ GPUs