AI-ENGINEERING
    intermediate

    Agents & Memory Systems

    AI Engineer path · lesson 85 of 124
    The idea
    The context window is the desk; memory is the filing cabinet. The desk is fast, small, and cleared overnight. Everything you want to survive to tomorrow has to be deliberately filed — and deliberately fetched back, which means something has to decide what is worth fetching.
    How to use
    Step through the conversation to fill the store, then switch to Recall, pick a query, and drag the recency weight to watch the ranking reorder.
    The same question, asked in a new session, with memory off and on.
    click to toggle
    Session 1 · last Tuesday
    user · I'm vegetarian, and I always fly out of Gatwick.
    Session 2 · today, fresh context
    user · Book me a flight to Lisbon and somewhere to eat when I land.
    agent · Happy to help — which airport are you departing from, and do you have any dietary preferences?
    Not a model failure. The facts were stated in a window that no longer exists, so from the model's side they were never said at all.
    this session only
    context window
    across sessions
    memory
    a retrieval policy
    what moves between them

    Key insight

    Memory is not a model capability — it is an architecture you build around the model. Every "the agent remembered" moment is a write policy, a store, and a retrieval step that somebody designed.

    Which memory does this belong in?

    Sort

    Sort each piece of state.

    Tap a piece, then tap where it goes — or drag it. Tap the tray to take one back.

    Short-term (this session)

    Drop items here

    Long-term (persisted)

    Drop items here
    6 pieces left to place

    Test your knowledge

    20 questions
    Question 1 of 20
    Score: 0

    The essential difference between a context window and memory is that the window:

    The short version

    The context window is the desk; memory is the filing cabinet. The desk is fast, small, and cleared overnight. Everything you want to survive to tomorrow has to be deliberately filed — and deliberately fetched back, which means something has to decide what is worth fetching.

    In more depth

    Agents & Memory Systems

    An agent's context window is not memory — it is a desk that gets cleared between sessions and overflows during long ones. Memory is the store outside the window plus the policy for moving things in and out.

    Four kinds

    Working — the context window itself. Bounded, volatile, and the only one the model can actually read.
    Semantic — durable facts. Overwrite on contradiction.
    Episodic — things that happened, with a timestamp. Append — both events happened.
    Procedural — how to do things here; learned corrections. Overwrite, and log the old rule.

    Getting overwrite vs append backwards either erases history you needed or accumulates contradictory preferences the agent picks between at random.

    Four operations

    1.Write — decide what is worth keeping. Writing everything = writing nothing.
    2.Retrieve — rank and inject a bounded top-k. This is the operation that costs tokens.
    3.Consolidate — merge, dedupe, resolve contradictions. Most often omitted; the reason long-lived agents degrade.
    4.Forget — expiry, decay, deletion. Needed for correctness, cost, and the user's right to erasure.

    Retrieval scoring

    score = relevance × (1 − w) + recency × w × 0.45 + importance × 0.25 — the weights are the ones the visualization actually computes, so the on-screen scores are reproducible by hand. Recency is a useful tiebreaker and a terrible primary signal — but it is the default in many implementations because it needs no embedding.

    Failure modes

    The store becomes a transcript · stale preferences outlive their truth · retrieved memory trusted as instruction · silent poisoning · cross-user leakage · unbounded cost.

    Interview Questions

    OpenAI
    Anthropic
    Google
    Meta
    Anthropic
    Microsoft
    Amazon
    OpenAI
    Google
    Meta
    Amazon
    Anthropic
    Netflix

    Love this visualization?

    Sign up to track your progress, earn achievements, use the AI assistant, and access your personal learning dashboard.

    100% free • No credit card required