LLMKube's 66× speedup is actually 35.4×

Two multipliers in LLMKube’s published GKE benchmark table are inconsistent with the raw figures on the same row: 1,026 ÷ 29 = 35.4×, not the stated 66×, and 64 ÷ 4.6 = 13.9×, not the stated 17×. Both are derivable by hand from numbers already in that table, with no missing inputs.

What LLMKube reports and under what conditions

LLMKube is a Kubernetes operator for heterogeneous LLM inference across NVIDIA CUDA, AMD Vulkan, and Apple Silicon. Its published benchmarks cover two hardware contexts: a GKE node with an NVIDIA L4 (CPU-path versus GPU-path), and a local dual RTX 5060 Ti with automatic layer sharding. Model and quantization are unspecified for both.

Metric	CPU (GKE, L4 node)	NVIDIA L4 (GKE)	LLMKube stated speedup	Derived speedup
Token generation	4.6 tok/s	64 tok/s	17×	13.9×
Prompt processing	29 tok/s	1,026 tok/s	66×	35.4×
Total response time	10.3 s	0.6 s	17×	17.2×
Model	unspecified	unspecified	—	—
Quantization	unspecified	unspecified	—	—

The response-time 17× checks out (10.3 ÷ 0.6 = 17.2×). Per-model results for the RTX 5060 Ti, with Q4 GGUF sizes pulled from our prior field note on bandwidth-bound decode as a reference denominator:

Model	Params	tok/s	P50	P99	Q4 GGUF size†	Implied BW_eff if Q4†
Llama 3.2 3B	3B	53.3	1930 ms	2260 ms	1.8 GB	95.9 GB/s
Mistral 7B v0.3	7B	52.9	1912 ms	2071 ms	4.2 GB	222.2 GB/s
Llama 3.1 8B	8B	52.5	1878 ms	2178 ms	—	—

†Q4 GGUF sizes from our prior field note, validated there to <2% error. LLMKube does not disclose quantization for these runs.

Analysis

The speedup column errors are arithmetic, not interpretive. The 17× token-generation figure matches the response-time ratio (17.2×) exactly. That points to the wrong row being sourced when the table was assembled, not a measurement problem. The prompt-processing discrepancy is larger: 66× versus a derived 35.4× is a 1.86× inflation. A team sizing GPU allocation on the premise that prompt processing is 66× faster than CPU will project roughly half the required GPU headcount. The raw tok/s figures (4.6, 29, 64, 1,026) may be accurate; the ratio column appears independently populated.

The flat ~53 tok/s plateau across 3B to 8B violates the bandwidth-bound expectation at uniform quantization. From our prior work, the bandwidth-bound decode identity:

$$\text{tok/s} \approx \frac{\text{BW}_{\text{eff}}}{M_{\text{GGUF}}}$$

holds to under 2% error across Raspberry Pi 5, Jetson Orin Nano, and Apple M4 Max. Rearranging to solve for implied effective bandwidth from the LLMKube figures:

$$\text{BW}_{\text{eff}} = \text{tok/s} \times M_{\text{GGUF}}$$

Substituting the 3B row at Q4:

$$\text{BW}_{\text{eff,3B}} = 53.3 \times 1.8 = 95.94 \text{ GB/s}$$

Substituting the 7B row at Q4:

$$\text{BW}_{\text{eff,7B}} = 52.9 \times 4.2 = 222.18 \text{ GB/s}$$

The ratio is 222.18 ÷ 95.94 = 2.31. Fixed hardware in a bandwidth-bound regime should produce 1.0. Three explanations fit: the 3B model is quantized more coarsely (larger GGUF, narrowing the size gap to the 7B); the inter-GPU transfer from LLMKube’s layer sharding between two GPUs becomes the binding constraint before either card’s bandwidth limit is reached; or context length or concurrency differs between runs, putting both in a fixed-overhead-dominated regime. LLMKube frames “consistent ~53 tok/s with automatic layer sharding” as a feature. It may equally be a sharding overhead floor.

The Metal Agent architecture is a hard constraint, not an optimization. LLMKube states that Metal GPU cannot be accessed from inside a container. The Metal Agent therefore runs as a native macOS process, watches the Kubernetes API for InferenceService resources tagged accelerator: metal, spawns llama-server natively, and registers its endpoints back into Kubernetes. That is the only viable path to GPU-backed inference on macOS in a Kubernetes fleet. A container-resident agent on that node falls back to CPU—LLMKube’s own GKE table shows what that costs: 4.6 tok/s versus 64 tok/s. The Metal Agent consequently lives outside Kubernetes’ restart domain. A crashed native process leaves a registered endpoint returning connection errors rather than triggering a pod restart, and LLMKube does not document how endpoint deregistration is handled in that failure mode.

The fail-closed PII/PHI routing claim has two distinct enforcement layers. LLMKube reports apply-time validation that rejects CRD rules routing PII/PHI-tagged traffic to cloud-tier backends, and a runtime enforcement path that returns HTTP 503 if the local pool cannot serve. The apply-time path is a Kubernetes admission webhook and is inspectable: it either rejects the manifest or it doesn’t. The runtime 503 is correct for regulated workloads but creates a capacity cliff with no degraded-mode fallback, by design. LLMKube also reports per-request audit logging covering rule, backend, tier, resolved timeout, and outcome. At high inference throughput the audit write path is an unconstrained latency sink, and no measurements of logging overhead are published.

What cannot be concluded

The GKE and RTX 5060 Ti benchmarks both omit model identifier, quantization level, and concurrency. Multiplier errors in the speedup column do not prove the raw tok/s figures are wrong—the denominators might be accurate while the ratio column was sourced elsewhere. The Q4 GGUF sizes used above come from our prior field note, not from LLMKube, so the implied-bandwidth calculation is only valid if LLMKube’s benchmarks also used Q4. Without disclosed quantization, the 2.31× implied-bandwidth discrepancy is compatible with multiple explanations; none can be ruled in or out.

What this means for Biquadia

The Metal Agent’s native-process constraint is structurally identical to the environment Biquadia already targets: Metal and ANE inference on Apple Silicon runs outside any container abstraction. Our prior field note on Orion’s ANE path measured 172.4 tok/s for GPT-2 124M at fp16 on ANE—already above LLMKube’s reported 64 tok/s for an NVIDIA L4 on a model and quantization that remain unspecified. For the llama-server path LLMKube’s Metal Agent uses, the bandwidth-bound ceiling on M4 Max class hardware (400 GB/s) at Q4 is 400 ÷ 1.8 = 222.2 tok/s for a 3B model, per our prior work—well above the ~53 tok/s plateau on dual RTX 5060 Ti hardware. The fail-closed 503 semantics for local-only inference are the correct routing policy for any Biquadia pipeline that processes user audio. A request that cannot be served on-device should fail explicitly, not transparently reroute to a remote endpoint.

---

Method: this note was drafted by us.anthropic.claude-sonnet-4-6 from two sources — the published README of defilantech/LLMKube and this studio’s own published measurements, linked above. Before publication an automated gate re-checked every extracted claim against the source document (13 claim(s) and 18 quantity(ies) verified) and every claim-shaped number in the draft against that evidence (12 derived from it, 23 row(s) supplied from our own tables). The studio has not re-run defilantech/LLMKube’s benchmarks; figures attributed to it are its own.

FAQ

Does the bandwidth-bound model apply to LLMKube's RTX 5060 Ti benchmarks?

It should at steady-state single-stream decode, and our prior work validates it to under 2% error on comparable hardware. The problem is that LLMKube does not disclose quantization, so there is no way to supply M_GGUF without assuming it. Using Q4 sizes from that prior field note produces implied effective bandwidths that differ by 2.31× between the 3B and 7B rows on the same hardware—inconsistent with a pure bandwidth-bound system.

Is the stated 17× token-generation speedup recoverable?

Not as stated. The derived figure is 64 ÷ 4.6 = 13.9×. The 17× matches the response-time speedup (10.3 ÷ 0.6 = 17.2×) and appears to have been transposed into the wrong row. If LLMKube corrects the table, 13.9× is internally consistent with the raw figures and plausible for an L4 versus a CPU-only path. 17× is not.

How does Kubernetes health checking interact with the Metal Agent?

Standard Kubernetes liveness and readiness probes operate against pod containers. The Metal Agent runs as a native macOS process outside any pod, so a crashed agent leaves its registered endpoint active and returning connection errors—no pod restart is triggered. LLMKube states that the agent registers endpoints back into Kubernetes but does not document deregistration on failure or what supervises the native process if Kubernetes cannot.

Can LLMKube's Apple Silicon performance projections be derived from the published figures?

Not directly—LLMKube does not publish a Metal Agent throughput measurement. What can be derived: applying the bandwidth-bound model with M4 Max at 400 GB/s and a 3B Q4 GGUF at 1.8 GB gives a ceiling of 222 tok/s, versus the ~53 tok/s plateau LLMKube reports for the same model class on dual RTX 5060 Ti. The comparison is hardware-to-hardware, not a LLMKube-to-LLMKube measurement, and actual throughput through llama-server with Metal will depend on quantization and batch configuration that LLMKube has not published.