Will this model fit in memory?
Estimate how much memory a local model needs: weights for each GGUF quantization, the KV cache for your context length, and which Mac memory size can run it.
- Weights
- KV cache
- Buffers
Estimate. Parameter counts and KV cache shapes come from each model's files on Hugging Face (read 2026-11-09). Weights use the average bits per weight of each GGUF type; buffers are a flat 1 GB. macOS lets the GPU use about three quarters of memory by default.
How the estimate works
Weights are the parameter count times the average bits per weight of the quantization. Q4_K_M stores about 4.9 bits per weight, Q8_0 about 8.5, F16 exactly 16.
The KV cache holds the keys and values for every token in the context: 2 × layers × KV heads × head size × context length × bytes per value. It grows linearly with context, which is why a model that fits at 8K can stop fitting at 128K. A Q8_0 cache roughly halves it.
Recent models change that formula, and the calculator follows each one's config: hybrid models such as Qwen3.8 Flash Next and Kimi K3 only cache their full-attention layers, Gemma 4 and gpt-oss cap their sliding-window layers at the window size, GLM-5 and Kimi cache one compressed latent per layer instead of full keys and values, and DeepSeek V4 stores most layers only every 4 or 128 tokens.
The model list is the popular models on Hugging Face, with parameter counts and attention shapes read from their published files. Mixture-of-experts models load every expert, so memory follows the total parameter count, not the active one.
Buffers for compute and scratch space depend on the runtime and batch size. The calculator adds a flat 1 GB; real numbers can be higher with large batches.
On a Mac, the GPU can use about three quarters of unified memory by default. The limit can be raised with sysctl iogpu.wired_limit_mb, at the cost of memory for everything else.
This is an estimate, not a measurement. Benchmarks on this site report measured peak memory next to it.