Embeddings & Vector Search
AI Engineer path · lesson 71 of 124Map text into a vector space, rank by cosine similarity, derive the cosine geometry interactively, trace the embedding pipeline, and see how nearest-neighbor search scales — the engine of semantic search and RAG.
Key insight
Meaning becomes geometry: similar text points the same direction, so retrieval is just “find the nearest vectors.” Cosine ranks by direction, top-k grabs the closest — that ranked list is exactly what RAG feeds the model.
From text to a ranked result
Order a semantic search, indexing first.
Tap a piece, then tap where it goes — or drag it. Tap the tray to take one back.
- 1Drop a step here
- 2Drop a step here
- 3Drop a step here
- 4Drop a step here
- 5Drop a step here
- 6Drop a step here
Test your knowledge
What does an embedding represent?
The short version
An embedding turns each sentence into an arrow on a compass: sentences about the same thing point the same way. Searching is just "which arrows point most like my question's arrow?"
In more depth
Embeddings & Vector Search
An embedding model maps text to a vector so semantically similar text points in similar directions (real embeddings typically run 100–10,000 dimensions: Cohere embed-english-light-3.0 is 384, CLIP 512, BERT base 768, OpenAI text-embedding-3-small 1536, text-embedding-3-large 3072). Cosine similarity = cos(angle) between two vectors — direction, not magnitude — and retrieval returns the top-k highest-cosine documents. At scale, an ANN index (HNSW/IVF) finds approximate neighbors fast.
Interview Questions
References & Further Reading
Love this visualization?
Sign up to track your progress, earn achievements, use the AI assistant, and access your personal learning dashboard.
100% free • No credit card required