Vector DB Internals & Serving
AI Engineer path · lesson 72 of 124Walk an HNSW small-world graph to a neighbor, descend its layer hierarchy, contrast IVF clustering + product quantization, and see the LLM-serving tricks: KV cache, batching, streaming.
Key insight
Exact search touches every vector; HNSW only follows edges toward the query, visiting a handful of nodes. Same neighbor, a fraction of the work — that’s what makes billion-vector search feasible.
What each index trades away
Sort each index type by what it gives up.
Tap a piece, then tap where it goes — or drag it. Tap the tray to take one back.
Exact — no recall loss
Drop items hereApproximate — trades recall for speed
Drop items hereTest your knowledge
An HNSW search always begins at which node?
The short version
Exact search is reading every name in the phone book; an ANN index is using the tabs and your sense of where "M" is to jump close, then look around. You might miss by one, but you finish a thousand times sooner.
In more depth
Vector DB Internals & Serving
Exact (flat) search compares the query to every vector — perfect recall but O(N). ANN indexes trade a little recall for big speedups: HNSW greedily walks a navigable small-world graph; IVF scans only the nearest clusters. Tune efSearch/nprobe to move along the recall↔latency curve.
Serving the LLM that reads results has its own tricks: the KV cache makes each new token cheap, batching raises throughput, and tokens are streamed as generated.
Interview Questions
References & Further Reading
Love this visualization?
Sign up to track your progress, earn achievements, use the AI assistant, and access your personal learning dashboard.
100% free • No credit card required