Task type splits MTP acceptance 2.05×

Our measurements put per-step draft acceptance at a lower bound of 0.615 for document tasks and 0.300 for reasoning tasks. Same quant level (Q8KP), same hardware (RTX 6000 Ada), same backend. A factor-of-two split the llama.cpp v0.2.0 README cannot surface — it contains no performance numbers. Task type governs the operating regime. Quant level and hardware, the variables the README’s feature list foregrounds, are secondary in what we can actually measure.

Claimed features, conditions absent

The llama.cpp v0.2.0 README makes no quantitative performance claims. All attributions here are from the README directly; the studio has not independently verified the structural claims.

It lists “1.5-bit, 2-bit, 3-bit, 4-bit, 5-bit, 6-bit, and 8-bit integer quantization for faster inference and reduced memory use.” No baseline, no hardware, no task type. It also states “CPU+GPU hybrid inference to partially accelerate models larger than the total VRAM capacity,” with the condition “model exceeds VRAM” but no throughput figure. The overarching framing is “LLM and VLM inference with minimal setup and state-of-the-art performance on a wide range of hardware” — no benchmark, no condition attached to “state-of-the-art.”

Speculative decoding and multi-token prediction do not appear in the README at all. The GGUF variant in the studio’s measurements (hauhaucs-aggressive-mtp) embeds dedicated draft heads; the MTP speedup figures below are properties of that specific model running inside llama.cpp, not of llama.cpp against an arbitrary checkpoint.

Task type as the governing variable

For speculative decoding with draft depth d and per-step acceptance probability α, the ceiling on speedup under zero verification overhead is:

$$S \leq 1 + d \cdot \alpha$$

Solving for the minimum acceptance consistent with an observed speedup:

$$\alpha \geq \frac{S - 1}{d}$$

Because verification is never free, this is a strict lower bound — actual acceptance must exceed the right-hand side. Substituting the studio’s Q8KP measurements for Standard MTP with stated d = 2:

• Document tasks: α ≥ (2.23 − 1) / 2 = 0.615
• Reasoning tasks: α ≥ (1.60 − 1) / 2 = 0.300

The document bound is more than twice the reasoning bound (0.615 / 0.300 = 2.05×). Quant level and hardware are identical across both rows.

The FastMTP measurements at Q3KP show the same directional split: the document-to-reasoning speedup ratio is 3.02 / 1.93 = 1.565, wider than the 2.23 / 1.60 = 1.394 ratio at Q8KP. The FastMTP draft depth is not disclosed in the studio’s measurement record, though. The FastMTP field note establishes that 3.02× requires draft depth ≥ 3; without a stated value, applying the bound above would require assuming an exact depth, which the data do not support.

Configuration	TG speedup — document	TG speedup — reasoning	Draft depth	Quant	Hardware	α lower bound (doc)	α lower bound (reas)
Baseline	1.00×	1.00×	—	Q8KP	RTX 6000 Ada	—	—
Standard embedded MTP	2.23×	1.60×	2 (stated)	Q8KP	RTX 6000 Ada	≥ 0.615	≥ 0.300
FastMTP	3.02×	1.93×	not disclosed	Q3KP	RTX 6000 Ada	not computable	not computable

The README’s claim that quantization delivers “faster inference” is not falsified by this data — lower-precision weights reduce memory bandwidth demand. But it cannot explain the document-versus-reasoning spread in the table, since both rows use the same quant.

A structural note: the README lists Metal, CUDA, HIP, MUSA, Vulkan, SYCL, AVX/AVX2/AVX512/AMX, NEON, and RVV as supported backends and instruction sets. The “wide range of hardware” framing explicitly declines to differentiate performance across them. The absence of backend-specific benchmarks is defensible given the combinatorial space, but it means the README provides no denominator for any of its performance language.

Derived in this analysis exact values· scroll →

0.615

Document tasks

0.300

Reasoning tasks

2.05×

The document bound is more than twice the reasoning bound

1.565×

The document-to-reasoning speedup

What cannot be concluded

The acceptance rate bounds apply only to the hauhaucs-aggressive-mtp GGUF variant, Q8KP, RTX 6000 Ada, and the specific document/reasoning task split the studio measured. They cannot be transferred to other checkpoints, the Metal backend, or lower quant levels holding MTP implementation fixed. The Q3KP FastMTP row cannot be reduced to acceptance rate bounds because the draft depth is not stated. The v0.2.0 release does not introduce new benchmark data anywhere in the document; it is a versioning event, not a performance claim.

What this means for Thumbdash

Thumbdash runs SmolLM2-360M-Instruct via llama.cpp’s Metal backend for keystroke-level completion — a task profile closer to document generation than chain-of-thought reasoning. If MTP draft heads were embedded in a model at this scale and served through the same path, the document-class acceptance regime (α ≥ 0.615) is the relevant prior. At 360M parameters, SmolLM2 fits entirely in Apple Silicon unified memory at any quantization level in the README’s listed range; the CPU+GPU hybrid path is not engaged, and Metal handles the full forward pass.

The operative metric for a speed-texting game is decode latency at batch size 1, not aggregate throughput — and the README is silent on both. Each accepted draft token in a document-class task represents a keystroke the user did not press. The factor-of-two gap between document and reasoning acceptance rates is precisely why task classification should precede any decision to invest in speculative decoding as a latency tool on this backend.

---

Method: this note was drafted by us.anthropic.claude-sonnet-4-6 from two sources — the published README of ggml-org/llama.cpp and this studio’s own published measurements, linked above. The claims and quantities it worked from were extracted from that document by qwen/qwen3.8-27b. Before publication an automated gate re-checked every extracted claim against the source document (10 claim(s) and 0 quantity(ies) verified) and every claim-shaped number in the draft against that evidence (5 derived from it, 30 row(s) supplied from our own tables). The studio has not re-run ggml-org/llama.cpp’s benchmarks; figures attributed to it are its own.

FAQ

Does the README's "no dependencies" claim conflict with listing Metal, CUDA, and Accelerate?

The README states "plain C/C++ implementation without any dependencies" and in the same description lists Metal, CUDA, HIP, MUSA, Vulkan, SYCL, and the Accelerate framework as backends or optimization targets. These are platform and driver dependencies. The studio has not verified how llama.cpp resolves this in practice; the "no dependencies" language likely refers to build-time third-party libraries rather than runtime platform APIs.

Why can the acceptance bound be computed for Standard MTP but not FastMTP?

The bound α ≥ (S − 1) / d requires a stated draft depth d. Standard MTP states d = 2; FastMTP's depth is listed as "not disclosed" in the studio's measurement record. The linked field note establishes depth ≥ 3 as a necessary condition for 3.02×, but using d = 3 in the formula would only be valid if the actual depth is exactly 3 — a larger value would produce a looser bound — and there is no data to choose between them.

Does a higher acceptance rate mean the draft heads are better trained?

Not exclusively. Acceptance rate depends on both how well the draft heads predict the main model's output distribution and how predictable the task itself is. Document-generation tokens have locally constrained continuation spaces; reasoning chains require non-obvious multi-step transitions. The same draft heads drop from ≥ 0.615 to ≥ 0.300 between task types with no change in weights — task structure alone shifts the acceptance regime by more than a factor of two.

Can these bounds transfer to the Metal backend Thumbdash uses?

No. The bounds are derived from measurements on RTX 6000 Ada via the CUDA backend. Metal and Apple Silicon unified memory represent a different compute and memory path; the VRAM-capacity constraint the README describes does not apply in the same form. The studio has not run equivalent MTP measurements on Metal, and no inference about whether the document-versus-reasoning split would widen, narrow, or hold is warranted.