3.02× FastMTP requires draft depth ≥ 3

The claimed 3.02× FastMTP document-TG speedup exceeds the hard theoretical ceiling for depth-2 speculative decoding, which is exactly 3.00×. That discrepancy is small enough to be measurement noise, but it cannot be reconciled with depth 2 at any acceptance rate. Draft depth must be at least 3 — a condition the model card never states, which means the published acceptance figures can’t validate the headline number.

Reported figures and their conditions

HauhauCS’s model card reports three distinct speedup configurations, each under conditions it does not uniformly specify:

Configuration	Document TG speedup	Reasoning TG speedup	Draft depth	Quant	Hardware
MTP disabled (baseline)	1.00×	1.00×	—	Q8_K_P	RTX 6000 Ada
Standard embedded MTP	2.23× (+123.4%)	1.60× (+59.6%)	2 (stated)	Q8_K_P	RTX 6000 Ada
FastMTP vs non-MTP	3.02×	1.93×	Not disclosed	Q3_K_P	RTX 6000 Ada
Full-window gate test	—	—	Not disclosed	Not disclosed	92.0% acceptance; 190,000-token cached prompt

The 35.2%/21.1% FastMTP-over-standard-MTP figures are ratios of the rows above: 2.23 × 1.352 ≈ 3.02 and 1.60 × 1.206 ≈ 1.93, which checks out with rounding on the individual multipliers. The 92.0% acceptance comes from a separate full-window gate test — 190,000 uncached prompt tokens plus 64 generated tokens, at 1,613.81 PP tok/s and 131.81 TG tok/s — not from either benchmark workload.

Analysis

The standard expected-tokens formula for speculative decoding with draft depth d and per-token acceptance probability α is:

$$E[\text{tokens per step}] = \frac{1 - \alpha^{d+1}}{1 - \alpha}$$

As α → 1, this converges to a hard ceiling:

$$\text{Speedup}_\text{max}(d) = \lim_{\alpha \to 1}\,\frac{1 - \alpha^{d+1}}{1 - \alpha} = d + 1$$

For d = 2 the ceiling is exactly 3.00×. No α ∈ [0, 1] produces a result above 3.00 at depth 2. The claimed FastMTP document speedup of 3.02× exceeds it. Small enough to sit within measurement variance — but irreconcilable with depth 2 at any acceptance rate. FastMTP’s draft depth must therefore be at least 3.

Substituting d = 3 and solving for α using the supplied document TG speedup of 3.02:

$$3.02 = \frac{1 - \alpha^4}{1 - \alpha} \implies 0.815^4 = 0.4412,\quad \frac{1 - 0.4412}{0.185} = 3.020 \implies \alpha_\text{doc} \approx 0.815$$

And for reasoning at d = 3, using the supplied 1.93×:

$$1.93 = \frac{1 - \alpha^4}{1 - \alpha} \implies 0.52^4 = 0.0731,\quad \frac{1 - 0.0731}{0.48} = 1.931 \implies \alpha_\text{reason} \approx 0.52$$

Back-solving the standard embedded MTP figures at the card’s stated depth of 2 gives: document α ≈ 0.72 (since 0.72³ = 0.3732, giving (1 − 0.3732)/0.28 = 2.238 ≈ 2.23) and reasoning α ≈ 0.42 (since 0.42³ = 0.0741, giving 0.9259/0.58 = 1.596 ≈ 1.60). Acceptance climbs going from standard MTP to FastMTP: 0.72 to ≥ 0.815 for document, 0.42 to ≥ 0.52 for reasoning. A deeper draft catches easier tokens; better-trained MTP heads likely help too.

The 92.0% full-window acceptance sits above both depth-3 estimates. That ordering is expected. A 190,000-token cached prompt tightly constrains the completion, while open-ended document and reasoning prompts produce far more diverse token sequences — so the full-window figure is an upper bound from an easier prediction task, not a proxy for benchmark acceptance.

The absolute TG figures — 138.37 document and 87.95 reasoning tok/s on Q3_K_P with FastMTP — are 23.5% and 3.9% faster than the Unsloth Q3 control. The gap between those two gains tracks the acceptance difference directly: at α ≈ 0.815, document tasks benefit substantially from the extra draft depth; at α ≈ 0.52, reasoning tasks don’t.

The layer composition adds a separate constraint. HauhauCS’s card reports 48 Gated DeltaNet layers and 16 gated-attention layers across the 64-layer stack. DeltaNet is a linear-recurrence variant with a fixed-size state matrix — not a growing KV sequence — so only the 16 attention layers contribute KV cache. At the 262,144-token native context, that 48:16 ratio (three stateful layers per attention layer) substantially limits KV growth compared to a pure-attention 27B model. The card doesn’t supply KV head counts, so the absolute footprint can’t be quantified from the available figures; the structural point is that 3 out of every 4 layers carry O(1) state in sequence length. That same heterogeneous-layer pattern appeared in the MoE coder reviewed at /blog/itria-hackernews-49321813.

What cannot be concluded

The depth-3 inference is a floor, not a confirmation. At depth 4, 3.02× would imply α ≈ 0.65 for document; at depth 5, α ≈ 0.55. The SHA-256-verified output match between FastMTP and standard embedded MTP confirms output identity on the tested prompts but carries no information about what draft depth was used. Without the card disclosing FastMTP’s depth, the acceptance rate can be bounded from below but not pinned.

What this means for itria

At 27B and Q3_K_P, this model’s weight file exceeds any iPhone or iPad memory envelope. The speculative decoding calibration is the transferable lesson, applicable at any model size. Task mode — document-like vs reasoning-like — determines acceptance rate; acceptance rate and draft depth together determine speedup via the formula above. Using the full-window acceptance rate, pulled from a highly constrained 190,000-token prompt, to project open-generation throughput will overestimate gains. Our prior measurements at /blog/biquadia-hackernews-49259339 established that token generation on consumer hardware is bandwidth-bound at the weight file; the speculative decoding multiplier sits on top of that baseline, so overestimating acceptance directly overstates the effective delivered tok/s. The DeltaNet hybrid architecture is also worth watching: if llama.cpp adds stateful-layer kernel support, a smaller model with a similar 3:1 state-to-attention ratio could relieve the long-context KV pressure that currently makes native contexts beyond ~8,000 tokens impractical on itria’s target hardware.

---

Method: this note was drafted by us.anthropic.claude-sonnet-4-6 from two sources — the model card for HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF and this studio’s own published measurements, linked above. Before publication an automated gate re-checked every extracted claim against the source document (15 claim(s) and 0 quantity(ies) verified) and every claim-shaped number in the draft against that evidence (24 derived from it, 21 row(s) supplied from our own tables). The studio has not re-run HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF’s benchmarks; figures attributed to it are its own.

FAQ

Does the 0/465 refusal result imply the model will not refuse under any prompt distribution?

HauhauCS's card does not describe the composition or provenance of the 465-prompt test set, so the zero-refusal figure measures coverage of that specific set only. Behaviour outside it cannot be inferred. Refusal-rate benchmarks are only as broad as their prompt distribution.

Can the 92.0% acceptance rate be used to project throughput on document or reasoning workloads?

No. HauhauCS's card reports that figure for a 190,000-token cached-prompt completion task, not for the document-TG or reasoning-TG benchmarks. The implied acceptance rates back-solved from the benchmark speedups are approximately 0.815 and 0.52 at depth 3 — substantially below 0.92. Use the task-specific speedup multipliers, not the gate-test figure, for projection.

Is the K_P quantization format compatible with unmodified llama.cpp builds?

HauhauCS's card does not state whether K_P quants use the standard llama.cpp format or require a custom build. The quality-improvement claim — "one or two quant levels at only around 5–15% more size" — is stated without a benchmark citation, so neither the quality delta nor the size overhead is independently verifiable from the supplied figures.

Does SHA-256 hash parity between FastMTP and standard embedded MTP outputs mean they are numerically identical in all cases?

HauhauCS's card states that every tested result reproduced the corresponding embedded-MTP output hash — strong evidence for the tested prompts. Speculative decoding's correctness guarantee applies because rejected draft tokens are replaced by the target model's distribution; the question is whether the tested prompts stressed the rejection path enough to cover divergent cases. Hash parity on the tested set is necessary but not sufficient for full equivalence across the token distribution.