Fun-ASR vLLM speedup: what conditions hide

Correction, 24 August 2026. An earlier version applied the single-H100 condition from the README’s batch_size_s example, and the tp=2 setting from a separate vLLM code example, to the 16x benchmark table. The table itself states neither hardware nor serving configuration. It also conflated the 720ms streaming SDK example with the separate WebSocket service. The original chart also placed speedup, accuracy, and error-rate percentages on one axis even though they measure different things. The conditions and the replacement time chart below now follow the README’s boundaries exactly.

The Fun-ASR README reports three speedup figures for vLLM over PyTorch on Fun-ASR-Nano. “3-5x faster” in a feature summary. “16x faster” in a benchmark annotation. The table entries 550s vs 34s compute to 550 ÷ 34 = 16.18x. A fourth figure—“~50% speedup over PyTorch”—belongs to a separate community codebase, Fun-ASR-vllm, under conditions the README never attaches.

Figures and their conditions

Inference throughput — two nearby 184-file measurements, not one fully specified setup

Method	Total time	RTFx	CER	Hardware	Batching
Default per-segment (PyTorch)	~583s¹	19.8	not reported	single H100	none—one VAD segment per call
PyTorch native	550s	21x	8.06%	not stated	not stated
batch_size_s=120 (PyTorch)	~363s²	31.8	not reported	single H100	VAD segments grouped to ~120s per call
vLLM (official)	34s	340x	8.20%	not stated	not stated
Fun-ASR-vllm (community)	not stated	not stated	not stated	not stated	not stated

¹ Derived: 11,539 ÷ 19.8 = 582.8s. The README uses 11,539s and explicitly says single H100 in the batch_size_s section. Its separate vLLM table says 11,541s but gives no hardware. The 2-second difference suggests related material; it does not establish identical hardware or configuration. ² Derived: 11,539 ÷ 31.8 = 362.9s.

Accuracy — Fun-ASR-Nano (0.8B) vs Fun-ASR full (7.7B)

Dataset	Fun-ASR-Nano WER	Fun-ASR full WER	Dataset type
AIShell1	1.80%	1.22%	README open-source benchmark section
AIShell2	2.75%	2.39%	README open-source benchmark section
Fleurs-zh	2.56%	2.53%	README open-source benchmark section
Fleurs-en	5.96%	4.74%	README open-source benchmark section
Librispeech-clean	1.76%	1.51%	README open-source benchmark section
Librispeech-other	4.33%	3.03%	README open-source benchmark section
WenetSpeech Meeting	6.60%	6.17%	README open-source benchmark section
WenetSpeech Net	6.01%	5.46%	README open-source benchmark section
Nearfield (industry)	7.79%	6.31%	README industry benchmark section; corpus details not stated
Farfield (industry)	5.79%	4.34%	README industry benchmark section; corpus details not stated
Complex Background (industry)	14.59%	11.45%	README industry benchmark section; corpus details not stated
English General (industry)	15.28%	13.73%	README industry benchmark section; corpus details not stated
Opensource (industry)	4.22%	3.38%	README industry benchmark section; corpus details not stated
Dialect (industry)	28.18%	15.21%	README industry benchmark section; corpus details not stated
Accent (industry)	12.90%	10.31%	README industry benchmark section; corpus details not stated
Lyrics (industry)	30.85%	21.00%	README industry benchmark section; corpus details not stated
Hiphop (industry)	30.87%	28.58%	README industry benchmark section; corpus details not stated
Industry average	16.72%	12.70%	9 industry rows above; corpus details not stated

All figures come from the Fun-ASR README. The studio has not reproduced them.

Where the comparison holds and where it breaks

16x and 340x/21x check out. 3-5x does not. 550 ÷ 34 = 16.18x; 340 ÷ 21 = 16.19x. Two independent ratios, both landing on the README’s “16x faster” callout to three significant figures. The “3-5x faster” phrase sits in a feature-summary line—“native high-throughput batch (3-5x faster) + WebSocket real-time streaming service”—with no dataset, no hardware spec, no batch configuration. Not derivable from any table in the document. Not comparable as stated; one of the two claims has no conditions.

Fun-ASR-vllm’s ~50% speedup (≈1.5x) and the official 16x are not the same comparison. The README calls Fun-ASR-vllm a community project with NVIDIA Triton integration. Hardware, audio length, batch size, error rate—none reported alongside the figure. The batch_size_s=120 optimisation on the PyTorch path alone gives 31.8 ÷ 19.8 = 1.606x. Close to the community’s 1.5x. Whether they’re the same effect is speculation, not derivation.

“Default per-segment” and “PyTorch native” are not established as the same baseline. The batch_size_s=120 claim uses per-segment decoding at RTFx=19.8 as its denominator—approximately 583s over 11,539 seconds of audio. The separate table’s “PyTorch native” row says 550s and RTFx=21x over 11,541 seconds. If those two near-matching corpus descriptions refer to the same material, swapping the denominator gives 31.8 ÷ 21 = 1.514x, not 1.6x. The README neither makes that identity explicit nor says what distinguishes the two PyTorch configurations.

The README reports both models side by side, but does not describe the evaluation harness. Its open-source benchmark section names AIShell, Fleurs, Librispeech, and WenetSpeech, making those rows inspectable in a way the industry section is not. It gives no corpus construction, normalisation rules, or speaker mix for the industry rows. The Nano-vs-full arithmetic within each row is clear; claiming controlled comparability or carrying the absolute values to another system would require details the README does not provide.

The “93%” claim and the Farfield industry row are different measurements. The README says recognition accuracy improves “to 93%” in far-field high-noise scenarios. The Farfield row shows Fun-ASR-Nano WER=5.79%, implying 100% − 5.79% = 94.21% accuracy. Same test material? Not stated. The 93% figure has no dataset reference, no normalisation procedure, no microphone spec. It doesn’t go in the table.

CER difference between vLLM and PyTorch native: real, small, consistent direction. 8.20% − 8.06% = 0.14 percentage points, same 184-file corpus. The README calls this “CER diff < 0.2%”. Correct. vLLM is fractionally worse; no cause stated.

What these numbers cannot support

The 16x figure is an offline batch result whose table omits hardware and serving configuration. A nearby example uses two-way tensor parallelism, but the README does not say the benchmark used that example. Its streaming SDK separately demonstrates 720ms chunks, while its WebSocket service is another mode again. FunASR 1.3.27 describes fallback behaviour after vLLM startup failures; 1.3.28 describes real-time stabilisation. Neither item supplies a replacement batch or streaming benchmark. The 16x number therefore does not transfer to streaming latency, a particular GPU count, or any named accelerator.

---

Method: this note was drafted by us.anthropic.claude-sonnet-4-6 from a single source — the published README of QwenAudio/Fun-ASR. The claims and quantities it worked from were extracted from that document by qwen/qwen3.8-27b. Before publication an automated gate re-checked every extracted claim against the source document (70 claim(s) and 66 quantity(ies) verified) and every claim-shaped number in the draft against that evidence (11 derived from it, no rows from our own tables were supplied to the draft). A post-publication human read corrected two cross-section condition joins and one broken internal link. The studio has not re-run QwenAudio/Fun-ASR’s benchmarks; figures attributed to it are its own.

FAQ

Does the 16x speedup apply to streaming use?

The 16x figure comes from an offline table covering 184 files and 11,541 seconds of audio. That table does not state its hardware or serving configuration. The README separately shows a streaming SDK example with 720ms chunks and a WebSocket service as another mode. Neither supplies a streaming-latency benchmark, so the batch number does not transfer to either path without its own measurement.

The "3-5x faster" and "16x faster" claims are both in the README. Which is right?

"16x faster" matches the table directly: 550 ÷ 34 = 16.18x. The studio hasn't reproduced this. "3-5x faster" has no measurement conditions. Can't be evaluated as stated. If it refers to different hardware, a shorter corpus, or a single-GPU run without tensor parallelism, it could coexist with 16x under genuinely different conditions. The README doesn't say that, though, so the two can't be reconciled from the document alone.

Is Fun-ASR-Nano competitive with the full 7.7B model?

In the README's open-source benchmark table the gap is narrow: 0.58 points on AIShell1 and 0.25 on Librispeech-clean. In its industry Dialect row it's 12.97 points (28.18% vs 15.21%). The Nano GGUF release goes down to approximately 484MB and runs on CPU with no Python dependency; the README does not document an equivalent GGUF path for the 7.7B model. Whether the gap is acceptable depends on whether your production audio resembles the named public sets or the undocumented industry test. For related context on model-size and quantisation tradeoffs, see Qwen3.8-27B Cold Fusion.

Are timestamps reliable from Fun-ASR-Nano?

No. The README states the released model.pt checkpoint does not include trained ctc_decoder.\* or ctc.\* weights, and that "any timestamp output is therefore not reliable." Speaker diarization needs an external pipeline. Neither Fun-ASR-Nano nor Fun-ASR-MLT-Nano emits speaker labels on its own.