DSpine on an M5 Ultra: Qwen3.8-27B from 43 to 134 tok/s

Qwen3.8-27B decodes at 43 tok/s on my Mac Studio M5 Ultra. With a 1.7 GB DSpine draft model attached, it decodes C++ at 133.7 tok/s. 3.1x, same machine, same quantization. At temperature 0 the output is identical token for token: the draft only changes the speed, not the text.

What DSpine is

DSpine is the successor to DFlash: a five-layer drafter that drafts a block of 16 tokens in parallel, with a code bridge and a transition table to keep the block causally consistent. The target verifies the block in a single pass and keeps the longest accepted prefix. Tencent released the draft weights; Ajay9o9 ported llama.cpp and quantized the draft. Upstream llama.cpp does not support DSpine yet, so you need his dspine branch.

The numbers

Idle machine, no other GPU work, median of 3 runs per case, 768 tokens generated, thinking mode off, exactly the protocol from the model card.

C++ generation133.7 · 3.1xCode (Python)127.7 · 2.9xMath118.7 · 2.7xLong prompt (13k)99.6 · 2.3xTemp 1 sampling99 · 2.3xProse52.5 · 1.2xBaseline (no draft)43.4 · every case, ±0.1

Structured content drafts well: code, math and C++ all accept around seven tokens per verify step. Prose is the worst case at 2.4, and it is also where the draft overhead eats most of the gain (1.2x). No run ever went below baseline.

Long prompts and temperature

A 12,989-token prompt still runs at 99.6 tok/s (2.3x). Acceptance drops to 5.2 tokens per step, the draft has less to latch onto once the context is dense. At temperature 1 acceptance drops to 5.6 and speed to 99.0 tok/s. The degradation is graceful: the draft never made a run slower than baseline.

Memory

Peak RSS: 19.8 GB without the draft, 23.3 GB with it. The draft costs about 1.7 GB in RAM. On 96 GB this is noise. On a 24 GB card it is a real trade-off: the model card reports ~64k usable context with DSpine against ~176k without speculation, because Qwen3.8 keeps a recurrent-state copy per drafted token.

M5 Ultra vs the RTX 3090

The same draft on a 24 GB RTX 3090, from the model card:

Two things fall out of this table. The 3090 and the M5 land within noise on speculative decode: the verify pass is compute-bound and both cards spend it at the same rate. The M5 wins on plain decode (43 vs 28 tok/s) thanks to memory bandwidth, which is why my speedup ratio looks smaller (3.1x vs the card’s 4.5x). The starting point is already faster. What transfers across hardware is the acceptance rate: 7.4 vs 7.6 tokens per verify on code, 6.7 vs 7.3 on math, 2.4 vs 2.5 on prose.

What I didn’t test

• DFlash2 on Metal, so no drafter comparison on this hardware
• Context beyond 13k input tokens
• Multi-slot serving or batched requests
• Quality at temperature > 0 (greedy output is identical by construction, so I only ran evals at temp 0)
• The 180B model I run daily. No DSpine draft exists for it yet

Reproduce

git clone -b dspine https://github.com/Ajay9o9/llama.cpp.git
cd llama.cpp
cmake -B build && cmake --build build --config Release -j

./build/bin/llama-server \
  -m Qwen3.8-27B-Q4_K_M.gguf \
  -md Qwen3.8-27B-DSpine-b16-Q4_K_M.gguf \
  --spec-type draft-dflash --spec-draft-n-max 15 \
  -ngl 999 --spec-draft-ngl 999 -fa on --jinja -c 8192 -np 1

Weights: aj9o9/Qwen3.8-27B-DSpine-b16-GGUF (draft, Apache-2.0) + any Qwen3.8-27B Q4_K_M target. Raw runs: dspine-m5-ultra.csv. Paper: arxiv.org/abs/2609.36173.