Local LLM model fit
Devstral Small 2 24B is a 24B Devstral model. This page estimates Q4 VRAM fit, Ollama command, context planning, and fallback choices for common local AI GPUs.
Software engineering agents, repo navigation, patch planning, and local coding workflows. Weakness: Large-context coding work is tight below 24GB VRAM.
| Hardware | Examples | Clean capacity | Q4 need | Status | Calculator |
|---|---|---|---|---|---|
| 6 GB VRAM entry GPU | GTX 1660, RTX 2060 6GB | 4.5 GB clean VRAM | 15 GB | RAM offload | Open calculator |
| 8 GB VRAM mainstream GPU | RTX 3060 Ti, RTX 4060, RTX 3070 | 6.5 GB clean VRAM | 15 GB | RAM offload | Open calculator |
| 10 GB VRAM older high-end GPU | RTX 3080 10GB | 8.5 GB clean VRAM | 15 GB | RAM offload | Open calculator |
| 12 GB VRAM local agent GPU | RTX 3060 12GB, RTX 4070, RTX 5070 | 10.5 GB clean VRAM | 15 GB | RAM offload | Open calculator |
| 16 GB VRAM creator GPU | RTX 4060 Ti 16GB, RTX 4080, RTX 5070 Ti, RTX 5080 | 14.5 GB clean VRAM | 15 GB | RAM offload | Open calculator |
| 24 GB VRAM homelab workstation | RTX 3090, RTX 4090 | 22.5 GB clean VRAM | 15 GB | Runs locally | Open calculator |
| 32 GB VRAM Blackwell workstation | RTX 5090 | 30.5 GB clean VRAM | 15 GB | Runs locally | Open calculator |
| 48 GB VRAM workstation | RTX A6000, L40S 48GB | 46.5 GB clean VRAM | 15 GB | Runs locally | Open calculator |
| Apple Silicon 32 GB unified memory | M2 Max 32GB, M3 Max 36GB | 26 GB unified | 15 GB | Runs locally | Open calculator |
| Apple Silicon 256 GB unified memory | Mac Studio M3 Ultra 256GB, Mac Studio M4 Ultra 256GB | 250 GB unified | 15 GB | Runs locally | Open calculator |
| Quantization | Estimated memory | Use case |
|---|---|---|
| Q4 / 4-bit | 15 GB | Default local inference balance |
| Q5 / 5-bit | 18.8 GB | Better quality, more VRAM |
| Q8 / 8-bit | 30 GB | High quality, much more VRAM |
| FP16 / 16-bit | 60 GB | Mostly workstation/server use |
This is a practical planning estimate, not a benchmark. Real memory use changes with backend, context length, KV cache, quantization file, drivers, and offloading settings.
20B · Local reasoning, agent planning, and tool-use workflows on 16GB+ GPUs
14B · Local coding, scripts, repo assistance, technical agents
14B · Local reasoning and debugging on 12GB/16GB GPUs
14B · Compact reasoning and technical assistant on 12GB/16GB GPUs