Qwen3.8-27B Cold Fusion: mixed attention

DavidAU has shipped GGUF quantizations of a fine-tuned Qwen3.8-27B focused on compressed thinking-token output. The architecture is not what most engineers will expect: of the 64 attention-style layers, 75% are Gated DeltaNet linear attention and only 25% are standard causal attention. KV cache memory scaling behaves materially differently from a conventional dense 27B.

What it is

These are NEO IMATRIX GGUFs of DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1, a fine-tune of Alibaba’s Qwen3.8-27B. The model card’s architecture spec reads 16 × (3 × (Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN)) — 48 of 64 blocks linear attention, 16 full causal. It carries a vision encoder (pipeline tag: image-text-to-text), supports English and Chinese, and is released under Apache 2.0.

Two GGUF families: standard and MTP (Multi-Token Prediction), the latter with an MTP suffix in the filename. For all quants, DavidAU sets the output tensor (10–20% of output) to 16-bit full precision and MTP tensors to Q8_0. Low-tier options are IQ4_XS and Q6_K.

Fact	Value	Condition / source
Parameters	27B	Model card
Layers	64 (48 Gated DeltaNet + 16 Gated Attention)	Model card architecture spec
Vision encoder	Yes	Model card pipeline tag image-text-to-text
Native context	262,144 tokens	Model card
Licence	Apache 2.0	Model card
Quant format	GGUF, NEO IMATRIX	DavidAU
Output tensor precision	16-bit full	DavidAU
MTP tensor precision	Q8_0	DavidAU
Low quant options	IQ4_XS, Q6_K	DavidAU
Thinking token reduction (median)	~2/3 vs base Qwen3.8	DavidAU, self-reported, no ablation protocol cited
4-bit vs BF16 retention	99%	DavidAU, self-reported, no methodology cited
NEO IMATRIX lift over standard GGUF	+2–4%	DavidAU, self-reported
Standard GGUF throughput, Q4_K_S	~75 t/s	DavidAU; RTX 5090, Windows 11, LMStudio
MTP GGUF throughput, Q4_K_S	>90 t/s	DavidAU; RTX 5090, Windows 11, LMStudio, 60% acceptance, 2-token predict
Self-assessed maturity vs Fable-Fusion-711	Level 1–2 vs level 7–8	DavidAU

Benchmark figures and what they actually show

The card provides mxfp8 and mxfp4 comparisons run in instruct mode against Qwen3.8-27B-Instruct, Qwen3.6-27B-Instruct, Qwen3.6-35B-A3B-Instruct, and Qwen3.5-27B-Instruct, across ARC-Challenge, ARC-Easy, BoolQ, HellaSwag, OpenBookQA, PIQA, and WinoGrande.

The fine-tune does clear Qwen3.8-27B-Instruct on most metrics. On mxfp8: arc/c 0.655 vs 0.591, arc/e 0.838 vs 0.782, obkqa 0.498 vs 0.448. The model card’s claim that the model “exceeds all Qwen 3.8, 3.6 and 3.5 27B critical core benchmarks” is not supported by its own numbers: Qwen3.6-27B-Instruct mxfp8 scores 0.910 on boolq vs Cold Fusion’s 0.898, 0.773 on hellaswag vs 0.751, and 0.742 on winogrande vs 0.738. The Qwen3.6 bar remains above on three of the seven metrics shown.

No BF16 baseline row appears in the card’s benchmark table. The card states “BF16 (full precision, 16 bit) will be roughly 2-5 points higher than MXFP8 in most metrics” — a general note, not a measured baseline for Cold Fusion specifically — so the “99% of BF16 at 4-bit” claim cannot be verified from the data in the card. The comparison benchmarks run at mxfp8/mxfp4 GPU precision formats — not GGUF quant levels — so they don’t directly test the quantizations being released here.

Where this is and is not worth trying

Agentic or multi-step reasoning loops where thinking-block length is the latency bottleneck. The model card reports a median ~2/3 reduction in thinking-block token count (range 1/2 to 1/10 vs base Qwen3.8); if it holds on your prompts, that compresses both time-to-first-token and total token cost in chained calls. Three reasoning modes — xhigh, medium, low — with documented sampling parameters give you a dial rather than a binary on/off. If the bottleneck is tool-call overhead or network round trips, thinking-token savings will not move your numbers.

Single-GPU desktop inference where you can meet the MTP conditions. The card shows ~75 t/s → >90 t/s on Q4_K_S at 60% acceptance, 2-token prediction, RTX 5090 — roughly 20% throughput gain under those specific conditions. Below 50% acceptance DavidAU advises switching to standard quants; temperatures above 1 or non-zero repetition penalty degrade acceptance rates. For a reference point on MTP depth tradeoffs in a similar model family, see the earlier note on 3.02× FastMTP draft depth requirements.

Vision-language tasks with local inference. The upstream Qwen3.8-27B is a native VLM; the pipeline tag carries through to this fine-tune. Locally-run sub-30B VLMs in GGUF form remain rare. The card does not benchmark vision tasks for the fine-tune specifically, so capability degradation relative to base is unquantified.

Not for: tasks requiring verified code or math benchmark scores for this specific fine-tune. The agentic benchmarks in the card — SWE-bench Pro 61.7, LiveCodeBench v6 90.3 — are upstream Qwen3.8-27B base figures, not results for this model. The GAIN training method has no cited paper and is described only qualitatively. The card rates this release at “level 1 or 2” relative to the same author’s more developed Fable-Fusion-711 pipeline at “level 7–8” — a candid self-assessment worth taking seriously before deploying in production.

What the card leaves open

The 99% BF16 retention figure at 4-bit is the central quality claim. No cited methodology, no ablation table. The benchmark comparisons at mxfp8/mxfp4 neither confirm nor deny it for GGUF quantizations. Whether the NEO IMATRIX calibration data matches the distribution of your target domain, or whether the thinking-token reduction figures hold outside the author’s test prompts, cannot be determined from the published card.

---

Method: this note was drafted by us.anthropic.claude-sonnet-4-6 from a single source — the model card for DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NM-DAU-NEO-MAX-MTP-GGUF. The claims and quantities it worked from were extracted from that document by qwen/qwen3.8-27b. Before publication an automated gate re-checked every extracted claim against the source document (80 claim(s) and 112 quantity(ies) verified) and every claim-shaped number in the draft against that evidence (2 derived from it, no rows from our own tables were supplied to the draft). The studio has not re-run DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NM-DAU-NEO-MAX-MTP-GGUF’s benchmarks; figures attributed to it are its own.

FAQ

Does the vision encoder survive the fine-tune and quantization?

The model card does not state whether the vision encoder weights were included in COLD FUSION training or held frozen, and it does not benchmark vision tasks for this fine-tune. The pipeline tag image-text-to-text is set and the architecture spec carries the vision encoder. Whether image understanding is degraded relative to base Qwen3.8-27B is untested in the published card.

When should I use MTP quants vs standard quants?

MTP quants are worth it when token acceptance stays above 50% with 2-token prediction; below that the model card advises switching to standard quants. DavidAU also warns to hold temperature at or below 1.0 and keep repetition penalty at 1.0 (off) — raising either degrades acceptance rate. The >90 t/s figure is specific to 60% acceptance on one machine; your domain prompts may land well below that threshold.

Is this the same pipeline as Fable-Fusion-711?

No. The model card explicitly rates this release at "level 1 or 2" versus Fable-Fusion-711 at "level 7–8." Fable-Fusion-711 involves 6 stages plus multiple sub-stages taking 7–10 days minimum; Cold Fusion GAIN V1.1 uses the lighter GAIN+Unsloth method. They share authorship and some underlying techniques, but this is a substantially earlier-iteration product.

What sampling settings does the card recommend for this model?

Thinking mode: temp=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, rep_penalty=1.0. Instruct/non-thinking mode: temp=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, rep_penalty=1.0. Minimum context window 16k; the card suggests 24k–32k even with reduced thinking blocks. For chat or roleplay, smoothing_factor=1.5 is recommended as an alternative to raising rep_penalty to 1.1–1.15.