# Presspeech's 100ms Latency Overhead Bandwidth calculation for Presspeech's Parakeet model on M4 Max shows inference takes ~1% of latency, with pipeline dispatch consuming the rest. Canonical page: https://makerportal.ai/blog/biquadia-github-1233899535 Author: Joshua Hrisko, Principal Engineer — MakerPortal Published: 2026-08-05 Section: Field note / On-Device AI · 7 min read Tags: biquadia, on-device-ai, metal-ane, privacy-arch --- Presspeech's reported ~100 ms latency is a system-level pipeline figure, not a model inference bound. Our published M4 Max memory bandwidth of 400 GB/s applied to presspeech's stated 500 MB macOS Parakeet model yields a bandwidth floor of 1.25 ms for a single forward pass — inference accounts for at most ~1% of the stated latency on the fastest Apple Silicon chip we have published bandwidth data for. ## What presspeech reports and under what conditions Presspeech states the following figures. Hardware SKU and audio clip length are not attached to most of them. | Metric | macOS | Windows | Conditions stated by presspeech | |---|---|---|---| | Transcription latency | ~100 ms | not reported | Hotkey-release to text at cursor; no hardware or clip-length qualifier | | Process RAM | ~80 MB | not reported | No hardware SKU; model assumed loaded | | CPU (idle) | 0% | not reported | Between dictations only | | Release artifact | 2.5 MB zip | — | Compressed bundle; relies on system frameworks | | Installed app | — | 4.4 GB | Self-contained; runtime bundled | | First-run model download | 500–600 MB | 2.5 GB | CoreML/FluidAudio format (macOS); format unspecified (Windows) | | Audio capture format | 16 kHz mono Float32 | 16 kHz mono Float32 | Stated in README for both platforms | ## Analysis **Where the 100 ms actually goes** Parakeet TDT v3 is a Token-and-Duration Transducer. In presspeech's push-to-talk mode, the full audio buffer is submitted after hotkey release, so the model runs one batch forward pass over all accumulated frames rather than decoding token-by-token. For a bandwidth-bound single forward pass, the time floor is: $$ t_{\text{bw}} = \frac{W}{B} $$ where $W$ is model weight bytes and $B$ is memory bandwidth in bytes per second. Substituting presspeech's macOS model lower bound and our published M4 Max figure: $$ t_{\text{bw}} = \frac{500 \times 10^6 \;\text{B}}{400 \times 10^9 \;\text{B/s}} = 1.25 \;\text{ms} $$ At the 600 MB upper bound, $t_{\text{bw}} = 1.5$ ms. The stated 100 ms is 67–80× above this floor. Running substantially below rated bandwidth doesn't change that conclusion — inference still cannot close that gap. Audio transfer is negligible by the same arithmetic. Presspeech states 16 kHz mono Float32 capture — Float32 is 4 bytes per sample: $$ r_{\text{audio}} = 16{,}000 \;\text{Hz} \times 4 \;\text{B/sample} = 64 \;\text{KB/s} $$ A 10-second utterance produces 640 KB, which the ANE ingests in roughly 1.6 µs at 400 GB/s. The overhead dominating the 100 ms budget is structural: AVFoundation has a minimum capture-buffer period that governs how quickly the final audio chunk flushes to the app after hotkey release; CoreML must schedule kernel dispatch through the system daemon before the ANE executes; CoreGraphics event injection carries its own round-trip. Presspeech publishes none of these per-stage figures. The 100 ms characterizes the user experience. It does not characterize the model. **The macOS/Windows model size gap** The macOS model downloads at 500–600 MB; Windows downloads 2.5 GB. Using presspeech's stated figures: $$ \frac{2{,}500 \;\text{MB}}{500 \;\text{MB}} = 5.0\times \qquad \left(\text{upper macOS bound: } \frac{2{,}500}{600} = 4.17\times\right) $$ A 4–5× size difference on the same Parakeet TDT v3 model is consistent with the macOS build shipping a CoreML-compiled quantized form targeted at the ANE, while the Windows build ships a higher-precision checkpoint alongside a bundled runtime. Presspeech does not state the quantization depth or on-disk format for either platform; this is an inference from the size gap alone. Total first-run acquisition on Windows is 4.4 + 2.5 = 6.9 GB against 500–602 MB on macOS. **The RAM figure and ANE memory accounting** Presspeech reports ~80 MB of process RAM while running a 500–600 MB model. On Apple Silicon, CoreML maps ANE-targeted model weights into memory owned by a system daemon; those bytes do not appear in the application process's footprint statistics. The ~80 MB is the process-visible footprint: app framework overhead, runtime heap, and activation tensors. The audio buffer adds almost nothing. At 64 KB/s, a 60-second capture buffer is 3.84 MB — under 5% of the stated figure. The ~80 MB is accurate for what it measures; model weights are simply not part of what it measures. **Cross-referencing the decode speed data** Our published autoregressive decode figures — 220 tok/s for a 3B Q4 model on M4 Max, 9.4 tok/s on a Raspberry Pi 5 (see [Running 3B LLMs on edge SBCs](/blog/running-3b-llms-on-microcontrollers)) — involve streaming model weights from memory once per output token. Parakeet TDT's single-pass batch inference streams weights once per utterance regardless of transcript length. The weight-streaming event happens once, not once per word or subword unit — which is why the bandwidth floor in absolute terms is so low.
Reported by rcourtman/presspeech ms
Transcription latency 100 ms Hotkey text insertion 100 ms

Chart: figures as rcourtman/presspeech reports them, drawn from the quantities this post cites. Bars are proportional to the reported values; the studio has not re-measured them.

## What cannot be concluded Presspeech does not specify which Apple Silicon SKU produced the ~100 ms and ~80 MB measurements, and our published bandwidth data covers M4 Max (400 GB/s) but not the lower-bandwidth chips elsewhere in the Apple Silicon family. On chips with less memory bandwidth, the bandwidth floor rises proportionally, but the pipeline-dominated conclusion holds as long as audio capture and CoreML dispatch overhead remain large relative to the inference floor — which will be the case unless presspeech publishes per-stage timing data that shows otherwise. The latency figure is also tied to an unspecified audio clip length; presspeech describes a single-sentence demo, and TDT forward-pass time scales with frame count, so longer dictations will extend it beyond the stated figure. ## What this means for Biquadia Biquadia loads CoreML neural enhancement and spatial audio models on Apple Silicon and must reason about latency budgets at block-processing timescales. Three observations fall out directly. The ~80 MB process RAM figure for a 500–600 MB model is not an anomaly — it is the expected CoreML/ANE behavior. Biquadia's capacity planning for co-resident models should use model download sizes as the denominator, not per-process footprint stats. The presspeech case shows the model occupies 500–600 MB outside the process footprint while the process itself reports only ~80 MB; planning against the latter number will underestimate total memory pressure. The 1.25–1.5 ms bandwidth floor bounds the forward pass contribution on M4 Max. For Biquadia's real-time chain, CoreML kernel scheduling and audio buffer handoff — not weight streaming — are the latency terms to optimize. The same compile-amortisation tradeoffs documented in [our Orion ANE measurements](/blog/biquadia-github-1171688936) apply: when a CoreML model is dispatched at block rate, per-dispatch overhead dominates bandwidth. The 4–5× macOS-to-reference size reduction inferred from presspeech's size gap is a calibration point for Biquadia model sizing. A neural model at full float precision will likely occupy roughly one-quarter to one-fifth of its reference checkpoint size after CoreML compilation for the ANE. Storage and initial download estimates should be made against the reference checkpoint before that compilation step, not after. --- *Method: this note was drafted by us.anthropic.claude-sonnet-4-6 from two sources — the published README of [rcourtman/presspeech](https://github.com/rcourtman/presspeech) and this studio's own published measurements, linked above. Before publication an automated gate re-checked every extracted claim against the source document (13 claim(s) and 10 quantity(ies) verified) and every claim-shaped number in the draft against that evidence (9 derived from it, 10 row(s) supplied from our own tables). The studio has not re-run rcourtman/presspeech's benchmarks; figures attributed to it are its own.* ## Questions this note answers ### Why is the Windows model download 4–5× larger than the macOS model if both use Parakeet TDT v3? Presspeech does not disclose the on-disk format or quantization depth for either platform. The most direct explanation for a 4–5× size ratio on the same model is the macOS build shipping a CoreML-compiled quantized form optimized for the ANE while the Windows build ships a larger-precision checkpoint. The Windows 4.4 GB install also bundles the entire inference runtime, whereas macOS relies on system-resident CoreML, AVFoundation, and AppKit — this independently accounts for the 2.5 MB vs 4.4 GB application size gap. ### Does the ~80 MB RAM figure mean the model fits in 80 MB at runtime? No. On Apple Silicon, CoreML routes ANE-targeted model weights into memory managed by the system daemon, not the application process. Presspeech's ~80 MB is the process-visible footprint — framework overhead, runtime heap, and activation tensors. The model parameters remain in the 500–600 MB region accounted to the system; tools that report per-process memory, including Activity Monitor and Instruments' footprint instrument, will not show those bytes in presspeech's row. ### Is ~100 ms tight enough for real-time neural audio processing? For push-to-talk dictation, yes — the user is not waiting on output concurrently with speaking. For Biquadia's block-processing DSP chain with strict latency budgets, a 100 ms system pipeline delay is not directly usable. The 1.25–1.5 ms bandwidth floor shows the ANE inference itself is compatible with low-latency block sizes; it is the audio capture pipeline, CoreML dispatch scheduling, and event injection overhead consuming the remaining 98+ ms that would each need to be instrumented and tightened individually. ### What does "0% CPU between dictations" actually imply architecturally? It means the hotkey listener is interrupt-driven, not a polling loop — a CoreGraphics event tap wakes the process only on the registered key event, leaving no scheduled work between dictations. The 0% is a floor on idle consumption only; presspeech does not publish a peak CPU figure for the active transcription window, during which AVFoundation capture, CoreML dispatch, and CoreGraphics event injection all draw cycles.