# PrivateRedact limits 7B decode to 4.0 tok/s PrivateRedact's auto-selection limits Ollama inference to 4.0 tok/s on an 8 GB Pi 5 by defaulting to CPU memory, ignoring unified bandwidth. Canonical page: https://makerportal.ai/blog/2026-08-11-itria-hackernews-49245161 Author: Joshua Hrisko, Principal Engineer — MakerPortal Published: 2026-08-11 Section: Field note / Local LLMs · 5 min read Tags: itria, local-llm, privacy-arch --- PrivateRedact publishes no throughput figures. The bandwidth-bound decode relationship we validated to within 2.1% error across six hardware/model pairings puts the 7B Q4 Ollama model — which the app can select automatically — at 4.0 tok/s on a 17 GB/s machine and 95 tok/s on an Apple M4 Max: a 23.75× range from identical weights, driven entirely by memory bandwidth. The app's "no cloud" headline has a stated carve-out the framing buries: the README notes that license verification calls Gumroad, making the activation event visible to a third-party processor regardless of document content. ## What PrivateRedact claims, and under what conditions The README states that "all processing is local — no file, hash, metadata, or content is transmitted anywhere," with one explicit exception: "license verification calling Gumroad." After activation, the app "works offline for a grace period." PDF export rasterizes each page and rebuilds from images, so no selectable source layer survives beneath redaction boxes. Using Ollama, the app "picks the model size automatically based on available memory," runs in regex-only mode without any model installed — at the cost of missing "semantic PII like names or diagnoses" — and applies Tesseract OCR with "a vision model double-check on low-quality scans." Audit logs "omit original PII by default" — the qualifier matters. No throughput, recall, precision, or F1 figures appear anywhere in the repository. ## Analysis The throughput gap is immediate. PrivateRedact delegates inference to Ollama, so its decode ceiling is governed by the bandwidth-bound formula: $$ \text{tok/s} \approx \frac{BW_{\text{GB/s}}}{M_{\text{GGUF (GB)}}} $$ Our prior field note ([watchOS LLM speed is capped by CPU bandwidth](/blog/itria-github-1051796718)) validated this formula against hardware ranging from Raspberry Pi 5 to Apple M4 Max, finding errors below 2.1% in every case. Applying it to the two model sizes most likely to appear in PrivateRedact's auto-selection — 3B Q4 at 1.8 GB and 7B Q4 at 4.2 GB — yields the following predictions, each confirmed against observed values from the same measurement set: | Hardware | Memory BW (GB/s) | Model | GGUF (GB) | Predicted tok/s | Observed tok/s | Measurement conditions | |---|---|---|---|---|---|---| | Raspberry Pi 5 (8 GB) | 17 | 3B Q4 | 1.8 | 9.44 | 9.4 | Single-board; no discrete GPU; itria field note | | Raspberry Pi 5 (8 GB) | 17 | 7B Q4 | 4.2 | 4.05 | 4.0 | Single-board; no discrete GPU; itria field note | | Jetson Orin Nano | 68 | 3B Q4 | 1.8 | 37.8 | 37 | Unified memory; embedded GPU; itria field note | | Jetson Orin Nano | 68 | 7B Q4 | 4.2 | 16.2 | 16 | Unified memory; embedded GPU; itria field note | | Apple M4 Max | 400 | 3B Q4 | 1.8 | 222 | 220 | Unified memory; Metal backend; itria field note | | Apple M4 Max | 400 | 7B Q4 | 4.2 | 95.2 | 95 | Unified memory; Metal backend; itria field note | For the 7B Q4 model at the two measurement extremes: $$ \text{RPi 5: } \frac{17}{4.2} = 4.05 \text{ tok/s} \qquad \text{M4 Max: } \frac{400}{4.2} = 95.2 \text{ tok/s} $$ The observed ratio is 95 / 4.0 = **23.75×** — same weights, same quantization, same Ollama backend. PrivateRedact's RAM-capacity-based auto-selection does not see this. Treating 32 GB of DIMM and 32 GB of unified memory as equally capable of running the 7B model is correct for capacity; DIMM bandwidth and unified-memory bandwidth are not the same number. The vision-model double-check on degraded scans compounds this. The README describes two-pass processing: Tesseract first, then a vision model on flagged pages — two separate Ollama inference calls. Co-resident models compete for the same bandwidth budget. The auto-selection logic accounts for the text model's GGUF footprint; whether it also accounts for the vision model being loaded simultaneously is not documented. The audit-log default deserves the same attention. "Omit original PII by default" implies a user-accessible toggle. Enable full logging and the audit file becomes a secondary PII store — its own retention, access-control, and deletion requirements, none of which the README addresses. Under GDPR or HIPAA, that configuration choice is a compliance question. ## What cannot be concluded The bandwidth formula bounds throughput. It says nothing about recall. PrivateRedact publishes no ground-truth evaluation of whether its hybrid regex + LLM pipeline catches the PII categories it targets. A 95 tok/s system that misses 20% of names in context is a materially different artifact from one that misses 2%, and nothing in the repository distinguishes those outcomes. "Semantic PII" — the term used to describe what the LLM adds over regex — covers a wide accuracy range; without an evaluation dataset and F1 figures, the LLM layer's contribution is unquantified. ## What this means for itria PrivateRedact's Ollama backend and itria's Metal-accelerated decode path hit the same bandwidth ceiling. On an M4 Max, that means 220 tok/s (3B Q4, observed); a PrivateRedact session through Ollama's Metal backend reaches the same figure from the same formula. The structural difference is validated latency. The 116-file ETOSCore test suite ([field note](/blog/itria-github-1051796718)) covers the inference pipeline in isolation, and our LLMKube measurements ([field note](/blog/biquadia-github-1095330682)) put Llama 3.2 3B at P99 2,260 ms. PrivateRedact has no published equivalent. For any studio feature touching document content, the Gumroad activation carve-out and the audit-log toggle are the two architecture decisions to resolve before production integration. --- *Method: this note was drafted by us.anthropic.claude-sonnet-4-6 from two sources — the published README of [monjurulkarim/privateredact](https://github.com/monjurulkarim/privateredact) and this studio's own published measurements, linked above. Before publication an automated gate re-checked every extracted claim against the source document (10 claim(s) and 0 quantity(ies) verified) and every claim-shaped number in the draft against that evidence (4 derived from it, 23 row(s) supplied from our own tables). The studio has not re-run monjurulkarim/privateredact's benchmarks; figures attributed to it are its own.* ## Questions this note answers ### Does the 23.75× throughput spread apply to typical laptop users? The spread is bounded by the hardware in the measurement set — RPi 5 at 17 GB/s and M4 Max at 400 GB/s. Most consumer laptops sit between those extremes. The key variable is unified vs. discrete GPU. On a discrete-GPU machine where the GGUF file exceeds VRAM capacity, Ollama layer-splits across GPU and CPU RAM; effective bandwidth falls to the CPU-RAM rate, making the formula accurate for the bottleneck tier but no longer the GPU's rated bandwidth. ### Is the Gumroad license call a meaningful document-privacy risk? Based on what PrivateRedact states, no: the call is for license key validation and the README explicitly says no document content is transmitted. The narrower risk is that the activation event traverses Gumroad's servers, making it visible to a third-party processor. Document exfiltration and activation telemetry are different threat models — but the "all processing is local" headline needs a footnote for any deployment with strict data-residency requirements. ### What does regex-only mode actually cover? The README states that regex-only mode will not catch "semantic PII like names or diagnoses," implying it does cover pattern-structured PII — phone numbers, email addresses, SSNs, credit card numbers, and similar regex-amenable formats. No exhaustive pattern list is published. Regex-only mode is a reasonable path when only structured PII matters and LLM latency is unacceptable; the coverage boundary just isn't formally documented. ### What happens when the offline grace period expires? PrivateRedact states the app "works offline for a grace period after activation" without specifying the duration. What happens when that period lapses without a successful Gumroad check is undocumented — whether processing stops, the app degrades to regex-only, or it prompts and waits. For air-gapped or intermittently connected deployments, confirm this with the vendor before relying on the app in production.