watchOS LLM speed is capped by CPU bandwidth

Our published figures confirm the bandwidth-bound decode formula across a 23.5× spread — 17 GB/s on a Raspberry Pi 5 to 400 GB/s on an M4 Max — with no fitted parameters and under 2.2% error at all six data points. That formula is the ceiling for ETOS-LLM-Studio on watchOS. The README states the watchOS runtime is fixed to the CPU path with no Metal acceleration, so CPU-accessible bandwidth divided by GGUF model size determines throughput before any on-device run.

What ETOS claims, with conditions

ETOS-LLM-Studio reports 349 Swift source files in its platform-agnostic ETOSCore framework, 155 in the iOS view layer, and 131 in the watchOS view layer — 635 production Swift files in total. The test suite adds 116 files and 41,055 lines. The largest single production file is Sync/WatchSyncManager.swift at approximately 1,540 lines — notable because the README notes the entire initial version was 1,800 lines.

Local model inference uses a Swift ↔ C ABI ↔ C++ bridge to llama.cpp, with libetos-llama.a precompiled via CMake + Ninja. On watchOS the runtime is “fixed to the CPU path” — Accelerate, no Metal. Encryption runs three layers deep: SQLCipher for the core database, AES-256-GCM (both simple password and PBKDF2 modes) for snapshots, and PBKDF2 + Keychain + biometrics for app lock. No throughput benchmarks appear in the README.

Analysis

The bandwidth-bound formula and the watchOS ceiling

For single-token autoregressive decode at batch size 1, each forward pass loads the full weight matrix once per token. The operation is memory-bandwidth-bound:

$$\text{tok/s}_{\text{predict}} = \frac{\text{BW}_{\text{GB/s}}}{M_{\text{GGUF (GB)}}}$$

Our edge SBC field note supplies six data points across three hardware targets — a complete set for verifying the formula:

Hardware target	BW (GB/s)	Model	$M_{\text{GGUF}}$ (GB)	Predicted tok/s	Observed tok/s	Error
Raspberry Pi 5 (8 GB)	17	3B Q4	1.8	17 ÷ 1.8 = 9.44	9.4	0.4%
Raspberry Pi 5 (8 GB)	17	7B Q4	4.2	17 ÷ 4.2 = 4.05	4.0	1.2%
Jetson Orin Nano	68	3B Q4	1.8	68 ÷ 1.8 = 37.8	37	2.1%
Jetson Orin Nano	68	7B Q4	4.2	68 ÷ 4.2 = 16.2	16	1.2%
Apple M4 Max	400	3B Q4	1.8	400 ÷ 1.8 = 222	220	1.0%
Apple M4 Max	400	7B Q4	4.2	400 ÷ 4.2 = 95.2	95	0.25%

Conditions for all six measurements are as published. No calibration constant. The residual stays under 2.2% across a 23.5× bandwidth span — consistent with near-complete saturation at batch size 1 across all three subsystems (LPDDR4X 32-bit, LPDDR5 128-bit, LPDDR5X unified).

For ETOS on watchOS, the formula becomes a statement about CPU-accessible bandwidth alone. Metal is not invoked; Accelerate handles the GEMV path. Whatever fraction of the Apple Watch SoC’s total bandwidth reaches the CPU — not the GPU — is the operative BW. To see where the interactive threshold sits, apply the formula at the lowest measured bandwidth:

$$\text{tok/s}_{\text{Pi5, 3B Q4}} = \frac{17}{1.8} = 9.44 \text{ tok/s}$$

The Pi 5 at 9.4 tok/s on a 1.8 GB model is already below comfortable streaming speed. Less CPU-accessible bandwidth than 17 GB/s means a lower figure for the same model. The formula is the bound; the missing input is the Watch SoC’s measured CPU bandwidth (see the limitation section below).

Code architecture ratios

ETOS’s self-reported file counts define the shape of a two-layer MVVM architecture:

Layer	Files	Share of 635 total	Condition
ETOSCore (business logic)	349	54.9%	Platform-agnostic; no UIKit/SwiftUI
iOS view layer	155	24.4%	iOS-specific SwiftUI views
watchOS view layer	131	20.6%	watchOS-specific SwiftUI views
ETOSCore tests	116	— (separate)	41,055 lines; not counted in 635

The watchOS view layer is 131 ÷ 155 = 84.5% the size of the iOS view layer by file count. If ETOSCore absorbs shared logic faithfully, the view layers should reflect only the rendering-surface difference between platforms. An 84.5% ratio implies either the watchOS UI carries near-iOS complexity, or some conceptually shared logic resides in platform-specific layers rather than ETOSCore. File counts alone can’t distinguish the two, but the ratio sets a lower bound on watchOS-specific engineering effort.

The test layer reports 41,055 ÷ 116 = 354 lines per test file on average. WatchSyncManager.swift at approximately 1,540 lines is 1,540 ÷ 354 = 4.35× that average — and 1,540 ÷ 1,800 = 85.6% of the entire v1 codebase. Cross-device sync state concentrated at that scale accretes test debt quickly. Each new sync edge case that lands in WatchSyncManager.swift has no structural pressure to decompose.

What cannot be concluded

ETOS does not publish CPU-accessible memory bandwidth for any Apple Watch SoC, and neither does this studio. The formula is confirmed for three external devices, but substituting into it requires a BW figure we don’t have. That the watchOS ceiling is low is provably correct from the formula structure; the specific tok/s value is not determinable from supplied figures. Separately, ETOS provides no inference benchmark for its iOS Metal path, so whether Metal acceleration on iPhone reaches throughput comparable to the M4 Max figures in our measurements is not established — the M4 Max data covers a Mac-class SoC, not the A-series chip in any shipping iPhone.

What this means for itria

Routing decode through Metal on iPhone gives itria access to the full unified-memory bandwidth rather than the CPU-side fraction. That’s exactly what the bandwidth-bound formula rewards. The M4 Max measurement (400 GB/s → 95 tok/s for a 4.2 GB 7B Q4 model) shows that saturating the full memory bus is what makes 7B models interactive on Apple silicon. ETOS on watchOS cannot do this — the constraint is architectural, not a product decision — so local-model parity between Watch and iPhone is unreachable for any model whose $M_\text{GGUF}$ keeps tok/s below interactive speed at the Watch’s CPU bandwidth.

For itria’s roadmap, the more actionable figure is the 131-file watchOS view layer. A watchOS companion is not a stripped-down iPhone view — ETOS’s architecture makes that explicit. Engineers scoping a watchOS itria surface should treat that 131-file count — built against a platform-agnostic ETOSCore that already absorbs most business logic — as an empirical lower bound on dedicated watchOS UI work for a client of similar feature depth. The RTX 5090 decode measurements and the Qwen3.6-35B active-parameter analysis matter for server-side inference itria may proxy to, but neither applies to the Watch CPU-only path ETOS documents.

---

Method: this note was drafted by us.anthropic.claude-sonnet-4-6 from two sources — the published README of Eric-Terminal/ETOS-LLM-Studio and this studio’s own published measurements, linked above. Before publication an automated gate re-checked every extracted claim against the source document (17 claim(s) and 9 quantity(ies) verified) and every claim-shaped number in the draft against that evidence (20 derived from it, 22 row(s) supplied from our own tables). The studio has not re-run Eric-Terminal/ETOS-LLM-Studio’s benchmarks; figures attributed to it are its own.

FAQ

Can ETOS actually run useful LLMs on Apple Watch locally?

ETOS reports local GGUF inference on watchOS via a CPU-only llama.cpp path. The formula tok/s = BW / M_GGUF governs throughput. Without a published CPU-accessible bandwidth figure for the target Watch SoC, no specific number is computable. The Pi 5 at 17 GB/s produces 9.4 tok/s on a 1.8 GB model — already below comfortable streaming speed. The formula is monotone in BW, so any Watch SoC with less than 17 GB/s CPU bandwidth scores lower on the same model.

Does the 84.5% watchOS-to-iOS file ratio suggest a problem with the architecture?

Not necessarily. It does mean the ETOSCore separation isn't collapsing the Watch client into a thin shell. If business logic is cleanly in ETOSCore, 131 watchOS view files could reflect genuine Watch-specific UI complexity — complications, always-on display states, Digital Crown interaction, and Workout session handling all require dedicated code. Whether that ratio reflects genuine UI work or leakage of shared logic into the view layer requires inspecting the files, not counting them.

How does ETOS's llama.cpp bridge relate to the imp backend in your decode measurements?

The imp backend our RTX 5090 field note covers delivers 42–48% decode improvement over baseline llama.cpp on an RTX 5090 under specific conditions (b9976, 2026-07-12, dense GGUF). ETOS bridges standard llama.cpp with no mention of imp or equivalent kernel replacement. Those imp figures are specific to sm120a and sm100 hardware. They don't apply to Apple silicon, and ETOS's iOS throughput depends on llama.cpp's Metal backend — a separate code path untouched by any of the RTX measurements.

Why does ETOS require Xcode 26.0+ when the current stable Xcode is 16.x?

ETOS's README states build requirements of Xcode 26.0+ and watchOS 26.0+ SDK. Those version numbers correspond to Apple's 2026 developer toolchain cycle — introduced under Apple's new major-version scheme and currently available as a beta. Building ETOS today requires that beta toolchain, not the current App Store Xcode release. Engineers should treat this as a moving target until the 26-series SDK ships as stable.