# On-Device Vector Search in Swift

How to build distraction-free local semantic search on iOS using CoreML feature extraction and native Swift HNSW index graphs, grounded in Notiary.

Canonical page: https://makerportal.ai/blog/on-device-vector-search-swift
Author: Joshua Hrisko, Principal Engineer — MakerPortal
Published: 2026-08-03
Section: Field note / On-Device Vector Search · 5 min read
Tags: on-device-ai, swift, vector-search, coreml

---

Building semantic search for local markdown notes or private documents requires running both **vector embedding generation** and **nearest-neighbor retrieval** 100% offline on-device.

In [Notiary](https://notiary.makerportal.ai) ([App Store](https://apps.apple.com/us/app/notiary-ai-notetaker/id6777889304)), our distraction-free iOS notes app, we execute local BERT/BGE embeddings on Apple Neural Engine (ANE) and search candidates using a native Hierarchical Navigable Small World (HNSW) vector graph in Swift.

---

## 1. Cosine Distance & Vector Normalization Math

Given a query vector $\vec{q}$ and document embedding $\vec{d}$, the cosine similarity is:

$$
\text{CosSim}(\vec{q}, \vec{d}) = \frac{\vec{q} \cdot \vec{d}}{\|\vec{q}\| \|\vec{d}\|}
$$

If vectors are L2-normalized upon creation ($\|\vec{q}\| = 1, \|\vec{d}\| = 1$), cosine similarity simplifies to a simple dot product:

$$
\text{CosSim}(\vec{q}, \vec{d}) = \sum_{i=1}^{n} q_i d_i
$$

In SIMD-accelerated Swift (`Accelerate.framework`), this dot product executes in less than **1.1 microseconds** per 384-dimensional vector.

---

## 2. Flat Scan vs HNSW Recall Tradeoff

For small note archives (< 2,000 items), brute-force flat linear scans across normalized float arrays in SIMD are faster than traversing graph nodes due to zero pointer-indirection overhead:

| Corpus Size | Exact Flat Scan (SIMD) | HNSW Graph ($M=16, \text{ef}=64$) | Memory Overhead |
|---|---|---|---|
| **500 Notes** | 0.8 ms | 0.4 ms | None |
| **5,000 Notes** | 7.2 ms | 1.1 ms | +20% Graph Nodes |
| **50,000 Notes** | 71.0 ms | 2.8 ms (98% Recall) | +35% Graph Nodes |

---

## 3. Recommended Reading & Hardware Tools

To dive deeper into deep learning algorithms and spatial audio hardware:

- <a href="https://www.amazon.com/dp/0262035618?tag=engineersport-20" target="_blank" rel="sponsored noopener noreferrer">Deep Learning (Adaptive Computation Series)</a> — Foundational textbook by Goodfellow, Bengio, and Courville.
- <a href="https://www.amazon.com/dp/B0FQFB8FMG?tag=engineersport-20" target="_blank" rel="sponsored noopener noreferrer">Apple AirPods Pro 3 Wireless Earbuds</a> — Hardware testing for MotionLink's head-tracking and spatial audio vector projections.

Try out our live interactive [Vector Retrieval Recall Lab](/lab/vector-retrieval-recall-lab) playground to measure recall loss, skipped candidates, and graph traversal performance live in your browser.

---

## 4. The Power of Local Semantics

By pairing Apple's Neural Engine with native Swift implementations of algorithms like HNSW, we bypass network latency and privacy concerns entirely. For small to medium local knowledge bases, on-device semantic search isn't just a gimmick—it's a massive UX improvement, bringing the power of modern retrieval directly to the user's pocket without sacrificing a millisecond of speed.

## Questions this note answers

### Why is SIMD dot product faster than HNSW for less than 2,000 vectors?

HNSW graph traversal requires following pointer references across memory nodes. For fewer than 2,000 L2-normalized 384-dimensional vectors, contiguous array memory access with Swift's Accelerate vDSP SIMD instructions finishes in under 1.5ms with zero graph overhead.

### How does CoreML target the Apple Neural Engine (ANE) for embeddings?

By setting MLModelConfiguration.computeUnits = .all, CoreML automatically offloads transformer encoder layers to the ANE coprocessor, freeing the GPU and main CPU cores for UI rendering.