Agents & Memory Systems
AI Engineer path · lesson 85 of 124Run a five-turn conversation that writes facts into a memory store, then fire queries at it and watch a relevance × recency × importance scorer decide which memories are worth the context tokens — and which it drops at exactly the wrong moment.
Key insight
Memory is not a model capability — it is an architecture you build around the model. Every "the agent remembered" moment is a write policy, a store, and a retrieval step that somebody designed.
Which memory does this belong in?
Sort each piece of state.
Tap a piece, then tap where it goes — or drag it. Tap the tray to take one back.
Short-term (this session)
Drop items hereLong-term (persisted)
Drop items hereTest your knowledge
The essential difference between a context window and memory is that the window:
The short version
The context window is the desk; memory is the filing cabinet. The desk is fast, small, and cleared overnight. Everything you want to survive to tomorrow has to be deliberately filed — and deliberately fetched back, which means something has to decide what is worth fetching.
In more depth
Agents & Memory Systems
An agent's context window is not memory — it is a desk that gets cleared between sessions and overflows during long ones. Memory is the store outside the window plus the policy for moving things in and out.
Four kinds
Getting overwrite vs append backwards either erases history you needed or accumulates contradictory preferences the agent picks between at random.
Four operations
Retrieval scoring
score = relevance × (1 − w) + recency × w × 0.45 + importance × 0.25 — the weights are the ones the visualization actually computes, so the on-screen scores are reproducible by hand. Recency is a useful tiebreaker and a terrible primary signal — but it is the default in many implementations because it needs no embedding.
Failure modes
The store becomes a transcript · stale preferences outlive their truth · retrieved memory trusted as instruction · silent poisoning · cross-user leakage · unbounded cost.
Interview Questions
References & Further Reading
Love this visualization?
Sign up to track your progress, earn achievements, use the AI assistant, and access your personal learning dashboard.
100% free • No credit card required