How much memory does Mistral Medium 3.5 need?
About 82 GB at Q4_K_M with an 8K context. It runs on a Mac with 128 GB of unified memory or more.
Try other settings in the calculatorMemory by quantization and context
Total memory: weights, KV cache in F16 and about 1 GB of buffers. Under each total, the smallest Mac that can run it with the default GPU memory limit.
| Quantization | 8K context | 32K context | 128K context |
|---|---|---|---|
| Q3_K_M | 66 GB96 GB Mac | 75 GB128 GB Mac | 111 GB192 GB Mac |
| Q4_K_M | 82 GB128 GB Mac | 91 GB128 GB Mac | 126 GB192 GB Mac |
| Q5_K_M | 95 GB128 GB Mac | 104 GB192 GB Mac | 139 GB192 GB Mac |
| Q6_K | 109 GB192 GB Mac | 118 GB192 GB Mac | 153 GB256 GB Mac |
| Q8_0 | 140 GB192 GB Mac | 148 GB256 GB Mac | 184 GB256 GB Mac |
| F16 | 259 GBtoo big for a Mac | 268 GBtoo big for a Mac | 304 GBtoo big for a Mac |
About the model
- Parameters
- 128B
- Type
- Dense
- Attention
- grouped-query attention
- KV cache, F16
- 3.0 GB at 8K, 47 GB at 128K
- On Hugging Face
- mistralai/Mistral-Medium-3.5-128B
Every layer keeps keys and values for the whole context, shared across groups of attention heads. The KV cache grows in a straight line with context.
Questions
How much memory does Mistral Medium 3.5 need?
About 82 GB at Q4_K_M with an 8K context: 78 GB of weights, 3.0 GB of KV cache and about 1 GB of buffers. At 32K context it needs about 91 GB.
Can Mistral Medium 3.5 run on a Mac?
Yes. At Q4_K_M with an 8K context it fits on a Mac with 128 GB of unified memory or more, using the default GPU memory limit of about three quarters.
How big is the Mistral Medium 3.5 KV cache?
About 3.0 GB at 8K tokens and 47 GB at 128K tokens with an F16 cache. Its attention design: grouped-query attention.
Other models
- Gemma 4 12B
- gpt-oss 20B
- Gemma 4 26B-A4B
- Qwen3.8 27B
- GLM-4.7 Flash
- Gemma 4 31B
- Qwen3.6 35B-A3B
- gpt-oss 120B
- Qwen3.8 Flash Next
- DeepSeek V4 Flash
- GLM-5.3
- DeepSeek V4 Pro
- Kimi K3
Data read from Hugging Face on 2026-11-09. These are estimates; benchmarks on this site report measured memory. How the estimate works. Compare all models in the model fit chart.
Get the next one
New benchmarks and agent runs by email, with the raw data. Or follow on RSS or X.