Presspeech's 100ms Latency Overhead

Presspeech’s reported ~100 ms latency is a system-level pipeline figure, not a model inference bound. Our published M4 Max memory bandwidth of 400 GB/s applied to presspeech’s stated 500 MB macOS Parakeet model yields a bandwidth floor of 1.25 ms for a single forward pass — inference accounts for at most ~1% of the stated latency on the fastest Apple Silicon chip we have published bandwidth data for.

What presspeech reports and under what conditions

Presspeech states the following figures. Hardware SKU and audio clip length are not attached to most of them.

Metric	macOS	Windows	Conditions stated by presspeech
Transcription latency	~100 ms	not reported	Hotkey-release to text at cursor; no hardware or clip-length qualifier
Process RAM	~80 MB	not reported	No hardware SKU; model assumed loaded
CPU (idle)	0%	not reported	Between dictations only
Release artifact	2.5 MB zip	—	Compressed bundle; relies on system frameworks
Installed app	—	4.4 GB	Self-contained; runtime bundled
First-run model download	500–600 MB	2.5 GB	CoreML/FluidAudio format (macOS); format unspecified (Windows)
Audio capture format	16 kHz mono Float32	16 kHz mono Float32	Stated in README for both platforms

Analysis

Where the 100 ms actually goes

Parakeet TDT v3 is a Token-and-Duration Transducer. In presspeech’s push-to-talk mode, the full audio buffer is submitted after hotkey release, so the model runs one batch forward pass over all accumulated frames rather than decoding token-by-token. For a bandwidth-bound single forward pass, the time floor is:

$$t_{\text{bw}} = \frac{W}{B}$$

where $W$ is model weight bytes and $B$ is memory bandwidth in bytes per second. Substituting presspeech’s macOS model lower bound and our published M4 Max figure:

$$t_{\text{bw}} = \frac{500 \times 10^6 \;\text{B}}{400 \times 10^9 \;\text{B/s}} = 1.25 \;\text{ms}$$

At the 600 MB upper bound, $t_{\text{bw}} = 1.5$ ms. The stated 100 ms is 67–80× above this floor. Running substantially below rated bandwidth doesn’t change that conclusion — inference still cannot close that gap.

Audio transfer is negligible by the same arithmetic. Presspeech states 16 kHz mono Float32 capture — Float32 is 4 bytes per sample:

$$r_{\text{audio}} = 16{,}000 \;\text{Hz} \times 4 \;\text{B/sample} = 64 \;\text{KB/s}$$

A 10-second utterance produces 640 KB, which the ANE ingests in roughly 1.6 µs at 400 GB/s. The overhead dominating the 100 ms budget is structural: AVFoundation has a minimum capture-buffer period that governs how quickly the final audio chunk flushes to the app after hotkey release; CoreML must schedule kernel dispatch through the system daemon before the ANE executes; CoreGraphics event injection carries its own round-trip. Presspeech publishes none of these per-stage figures. The 100 ms characterizes the user experience. It does not characterize the model.

The macOS/Windows model size gap

The macOS model downloads at 500–600 MB; Windows downloads 2.5 GB. Using presspeech’s stated figures:

$$\frac{2{,}500 \;\text{MB}}{500 \;\text{MB}} = 5.0\times \qquad \left(\text{upper macOS bound: } \frac{2{,}500}{600} = 4.17\times\right)$$

A 4–5× size difference on the same Parakeet TDT v3 model is consistent with the macOS build shipping a CoreML-compiled quantized form targeted at the ANE, while the Windows build ships a higher-precision checkpoint alongside a bundled runtime. Presspeech does not state the quantization depth or on-disk format for either platform; this is an inference from the size gap alone. Total first-run acquisition on Windows is 4.4 + 2.5 = 6.9 GB against 500–602 MB on macOS.

The RAM figure and ANE memory accounting

Presspeech reports ~80 MB of process RAM while running a 500–600 MB model. On Apple Silicon, CoreML maps ANE-targeted model weights into memory owned by a system daemon; those bytes do not appear in the application process’s footprint statistics. The ~80 MB is the process-visible footprint: app framework overhead, runtime heap, and activation tensors.

The audio buffer adds almost nothing. At 64 KB/s, a 60-second capture buffer is 3.84 MB — under 5% of the stated figure. The ~80 MB is accurate for what it measures; model weights are simply not part of what it measures.

Cross-referencing the decode speed data

Our published autoregressive decode figures — 220 tok/s for a 3B Q4 model on M4 Max, 9.4 tok/s on a Raspberry Pi 5 (see Running 3B LLMs on edge SBCs) — involve streaming model weights from memory once per output token. Parakeet TDT’s single-pass batch inference streams weights once per utterance regardless of transcript length. The weight-streaming event happens once, not once per word or subword unit — which is why the bandwidth floor in absolute terms is so low.

What cannot be concluded

Presspeech does not specify which Apple Silicon SKU produced the ~100 ms and ~80 MB measurements, and our published bandwidth data covers M4 Max (400 GB/s) but not the lower-bandwidth chips elsewhere in the Apple Silicon family. On chips with less memory bandwidth, the bandwidth floor rises proportionally, but the pipeline-dominated conclusion holds as long as audio capture and CoreML dispatch overhead remain large relative to the inference floor — which will be the case unless presspeech publishes per-stage timing data that shows otherwise. The latency figure is also tied to an unspecified audio clip length; presspeech describes a single-sentence demo, and TDT forward-pass time scales with frame count, so longer dictations will extend it beyond the stated figure.

What this means for Biquadia

Biquadia loads CoreML neural enhancement and spatial audio models on Apple Silicon and must reason about latency budgets at block-processing timescales. Three observations fall out directly.

The ~80 MB process RAM figure for a 500–600 MB model is not an anomaly — it is the expected CoreML/ANE behavior. Biquadia’s capacity planning for co-resident models should use model download sizes as the denominator, not per-process footprint stats. The presspeech case shows the model occupies 500–600 MB outside the process footprint while the process itself reports only ~80 MB; planning against the latter number will underestimate total memory pressure.

The 1.25–1.5 ms bandwidth floor bounds the forward pass contribution on M4 Max. For Biquadia’s real-time chain, CoreML kernel scheduling and audio buffer handoff — not weight streaming — are the latency terms to optimize. The same compile-amortisation tradeoffs documented in our Orion ANE measurements apply: when a CoreML model is dispatched at block rate, per-dispatch overhead dominates bandwidth.

The 4–5× macOS-to-reference size reduction inferred from presspeech’s size gap is a calibration point for Biquadia model sizing. A neural model at full float precision will likely occupy roughly one-quarter to one-fifth of its reference checkpoint size after CoreML compilation for the ANE. Storage and initial download estimates should be made against the reference checkpoint before that compilation step, not after.

---

Method: this note was drafted by us.anthropic.claude-sonnet-4-6 from two sources — the published README of rcourtman/presspeech and this studio’s own published measurements, linked above. Before publication an automated gate re-checked every extracted claim against the source document (13 claim(s) and 10 quantity(ies) verified) and every claim-shaped number in the draft against that evidence (9 derived from it, 10 row(s) supplied from our own tables). The studio has not re-run rcourtman/presspeech’s benchmarks; figures attributed to it are its own.

FAQ

Why is the Windows model download 4–5× larger than the macOS model if both use Parakeet TDT v3?

Presspeech does not disclose the on-disk format or quantization depth for either platform. The most direct explanation for a 4–5× size ratio on the same model is the macOS build shipping a CoreML-compiled quantized form optimized for the ANE while the Windows build ships a larger-precision checkpoint. The Windows 4.4 GB install also bundles the entire inference runtime, whereas macOS relies on system-resident CoreML, AVFoundation, and AppKit — this independently accounts for the 2.5 MB vs 4.4 GB application size gap.

Does the ~80 MB RAM figure mean the model fits in 80 MB at runtime?

No. On Apple Silicon, CoreML routes ANE-targeted model weights into memory managed by the system daemon, not the application process. Presspeech's ~80 MB is the process-visible footprint — framework overhead, runtime heap, and activation tensors. The model parameters remain in the 500–600 MB region accounted to the system; tools that report per-process memory, including Activity Monitor and Instruments' footprint instrument, will not show those bytes in presspeech's row.

Is ~100 ms tight enough for real-time neural audio processing?

For push-to-talk dictation, yes — the user is not waiting on output concurrently with speaking. For Biquadia's block-processing DSP chain with strict latency budgets, a 100 ms system pipeline delay is not directly usable. The 1.25–1.5 ms bandwidth floor shows the ANE inference itself is compatible with low-latency block sizes; it is the audio capture pipeline, CoreML dispatch scheduling, and event injection overhead consuming the remaining 98+ ms that would each need to be instrumented and tightened individually.

What does "0% CPU between dictations" actually imply architecturally?

It means the hotkey listener is interrupt-driven, not a polling loop — a CoreGraphics event tap wakes the process only on the registered key event, leaving no scheduled work between dictations. The 0% is a floor on idle consumption only; presspeech does not publish a peak CPU figure for the active transcription window, during which AVFoundation capture, CoreML dispatch, and CoreGraphics event injection all draw cycles.