algorithms.live

    AI engineering, visualized

    The parts of an LLM application in the order a request meets them: the prompt, the model call, retrieval, tools, the agent loop, then evaluation and serving.

    Start with what AI engineering is and prompt engineering. If transformers are new to you, read the LLMs and transformers hub first.

    Start with What Is AI Engineering? 57 topics

    Start here

    Beginner topics, in the order they build on each other.

    1. What Is AI Engineering?Compare ML engineering with AI engineering row by row, climb the adaptation ladder from a prompt edit to a pretraining run, and see which rung a problem actually belongs on.Beginner13m
    2. Planning AI ApplicationsDecide whether to build an AI feature at all — rank the three motivations, place it on Apple's three product axes and watch the engineering requirements fall out, earn each rung of Crawl-Walk-Run, and drag a slider along the last mile that LinkedIn actually measured.Beginner16m
    3. Prompt EngineeringToggle prompting techniques — system role, few-shot, chain-of-thought, output constraints — on a real task and watch the constructed prompt and the model's answer get sharper.Beginner13m
    4. Sampling & Decoding ParametersReshape a real next-token distribution with temperature, cut its tail with top-k or top-p, then sample repeatedly to feel how the knobs trade determinism for diversity — and why T=0 still isn't fully reproducible.Beginner15m
    5. Data SchemaValidate a record live, see the same schema in SQL, JSON Schema, Pydantic, and Protobuf, walk a backward-compatible migration, and use the schema to constrain an LLM's structured output.Beginner11m
    6. Multimodal LLMs (Vision)Slice an image into patches and watch it become tokens, follow attention flowing between words and pixels, and price what a screenshot really costs in context.Beginner14m

    Next

    Intermediate topics.

    1. Understanding Foundation ModelsSee what a foundation model is pretrained on, derive the compute-optimal split between model size and training tokens, and step the pretrain → SFT → preference-tuning pipeline that turns a text completer into an assistant.Intermediate18m
    2. The Multilingual Data GapCompare a language's share of the web against its share of humanity, price the same request in English and Burmese to see the tokenization tax, and find out why refusal behaviour is the first thing that breaks outside English.Intermediate16m
    3. Structured OutputsMake a model emit something a program can parse. Step the five layers from prompting to finetuning, watch a grammar mask the logit vector so an invalid token becomes unreachable rather than unlikely, and see how far LinkedIn's defensive parser carried them.Intermediate17m
    4. Prompt CachingSimulate API calls with and without prefix caching, compare provider pricing, and learn how to structure prompts for maximum cache hits — the cheapest performance optimisation in LLM serving.Intermediate14m
    5. Function / Tool CallingWalk a tool call end-to-end: the model emits a schema-constrained request, the runtime executes the function, and the result returns for a grounded answer — the foundation under agents and MCP.Intermediate16m
    6. Reasoning Models & Test-Time ComputeToggle extended thinking on a trick question, drag a thinking-budget slider along the test-time scaling curve, and price what those hidden tokens actually cost.Intermediate15m
    7. Evaluation Metrics: Perplexity & SimilarityCompute cross-entropy and perplexity from a predicted probability, then run four adversarial reference/output pairs through exact match, lexical overlap and embedding similarity and watch the three metrics disagree with each other and with a human.Intermediate17m
    8. LLM-as-a-JudgeWatch judge biases flip a winner, compare absolute/pairwise/reference scoring, tour the full bias catalog with mitigations, and calibrate the judge against humans before trusting it.Intermediate16m
    9. Comparative Evaluation & LeaderboardsCast votes in a live three-model arena and watch Elo ratings separate, read a rating gap as a win probability, then size the experiment: how many votes it actually takes to tell two close models apart.Intermediate16m
    10. Instruction-Following EvaluationA model can read a tweet correctly and still answer HAPPY when you asked for NEGATIVE, POSITIVE or NEUTRAL. Step IFEval's 25 automatically verifiable instruction types, score one with INFOBench's yes/no criteria, and see why roleplaying needs negative knowledge.Intermediate17m
    11. Public Benchmarks & ContaminationRead a leaderboard properly. Compare two suites that share two benchmarks out of ten, spot which ones correlate at 0.90 and are counting the same capability twice, and watch a headline score inflate while the clean-samples score never moves.Intermediate19m
    12. Model SelectionRun model choice as a four-stage funnel — hard gates, public evidence, your own eval, then economics — weigh a hosted API against open weights, watch benchmark contamination inflate a score, and find the volume where self-hosting starts to pay.Intermediate17m
    13. Semantic LayerDefine a business metric once; a dashboard, a SQL API, and an LLM analyst all read the same governed definition. Change it once and every consumer updates — the source of truth for AI analytics.Intermediate13m
    14. Graph DatabaseClick through the property-graph model, run Cypher-style traversals compared to the equivalent SQL joins, and see illustrated use cases — the engine behind recommendations and knowledge graphs.Intermediate15m
    15. Intelligent Document Processing (IDP)Turn a PDF into structured data: walk the pipeline stage-by-stage (ingest → OCR → layout → schema extraction → validation → HITL), then design confidence-driven human-in-the-loop routing that scales.Intermediate17m
    16. Embeddings & Vector SearchMap text into a vector space, rank by cosine similarity, derive the cosine geometry interactively, trace the embedding pipeline, and see how nearest-neighbor search scales — the engine of semantic search and RAG.Intermediate14m
    17. Hallucinations: Causes & MitigationWatch a model confidently continue a false premise, then stack the standard mitigations — grounding, tools, citations, abstention, self-consistency, verifier — and see the residual-risk meter fall.Intermediate15m
    18. RAG (Retrieval-Augmented Generation)Run retrieve → augment → generate as an animated pipeline, see why grounding beats a frozen model when facts change, peek at the offline indexing half, and diagnose where RAG breaks.Intermediate16m
    19. RAG vs Fine-tuning vs PromptingScore the three approaches against your requirements, see which layer of the stack each one changes, compare build/per-call/update costs, and learn how they combine in production.Intermediate14m
    20. LangChainRun a prompt→model→parser→tool chain, learn the LCEL pipe and the Runnable interface (invoke/stream/batch), tour the component toolbox, and see why loops push you to LangGraph.Intermediate15m
    21. Model Context Protocol (MCP)Step the MCP lifecycle, see what a server exposes (tools/resources/prompts and who controls each), watch N×M integrations collapse to N+M, and weigh the transport + security model.Intermediate16m
    22. AI Agent Loop (ReAct)Watch an agent reason and act in a loop, see what one step really is (context in, action out), learn how the loop stops safely, and compare ReAct with plan-execute, reflexion, and tree-of-thought.Intermediate15m
    23. Context CompactionWhen a conversation context window fills up, compaction replaces verbose history with a concise rolling summary. Learn how compaction works, what survives vs. what gets dropped, and how to pair it with prompt caching for maximum efficiency.Intermediate14m
    24. Agents & Memory SystemsRun a five-turn conversation that writes facts into a memory store, then fire queries at it and watch a relevance × recency × importance scorer decide which memories are worth the context tokens — and which it drops at exactly the wrong moment.Intermediate17m
    25. Dataset EngineeringMove three sliders to find whether quality, coverage or quantity is your binding constraint, weigh four data sources against their characteristic failures, run a synthesis loop with and without a verifier, and toggle a processing pipeline to see how much of a raw pile survives.Intermediate18m
    26. Model Routing & Cost EngineeringStream a live request mix through one of four policies — all-big, all-small, heuristic router, or confidence cascade — and watch cost, latency, and quality shift as you switch between them. A hands-on way to explore the tradeoffs every LLM product owner argues about.Intermediate16m
    27. Jev & System One ModelsMeet Jev, TypeSafe's model that never writes text: send one state and a set of typed questions, and every answer comes back at once with a probability. Compute confidence by hand, read a reliability diagram, and set the thresholds that decide when code acts and when a person looks.Intermediate16m
    28. LLM EnsemblingCombine multiple model calls into one better answer — self-consistency voting, Mixture-of-Agents, best-of-N reranking, weighted/stacked ensembling, and multi-agent debate — and see the cost/quality tradeoff each one makes.Intermediate21m
    29. Prompt Management & LLMOpsVersion prompts like code: promote from dev to prod through an eval gate, roll out safely with a canary, and close the loop from production feedback back into the next version.Intermediate17m
    30. AI Architecture & the Feedback LoopBuild an AI application architecture one layer at a time — model, context, guardrails, router, cache, observability — each justified by the failure that forces it, then rank the explicit and implicit user-feedback signals that refill your eval set.Intermediate17m
    31. Voice AI: ASR, TTS & RealtimeTune each stage of the ASR → LLM → TTS pipeline against the ~1.2-second conversational bar, trigger a barge-in, and compare cascades with native speech-to-speech models.Intermediate15m
    32. Image Generation & DiffusionDrag a slider to watch an image emerge from pure noise, tour the latent-diffusion architecture, and tune guidance and step count against quality and cost.Intermediate15m

    Advanced

    For when the rest feels solid.

    1. Ontology & Knowledge GraphRepresent facts as RDF triples over a typed graph, then run a reasoner to derive a new implicit relationship — the structured, connected knowledge behind GraphRAG.Advanced16m
    2. Vector DB Internals & ServingWalk an HNSW small-world graph to a neighbor, descend its layer hierarchy, contrast IVF clustering + product quantization, and see the LLM-serving tricks: KV cache, batching, streaming.Advanced18m
    3. Advanced RAGToggle retrieval upgrades and watch the right chunk climb, fuse keyword + vector search with RRF, funnel candidates through a cross-encoder re-ranker, and transform vague queries (rewrite, HyDE, multi-query).Advanced18m
    4. GraphRAGWatch flat retrieval fail a multi-hop question, traverse a knowledge graph to connect the dots, see how an LLM extracts the graph as triples, and compare local (traversal) vs global (community-summary) queries.Advanced17m
    5. LangGraphWalk a stateful agent graph, see how channel reducers merge state, route with conditional edges and cycles (with a recursion limit), and get durable execution: checkpoints, resume, and human-in-the-loop.Advanced17m
    6. Multi-Agent SystemsRun an orchestrator that decomposes, delegates, and aggregates; compare supervisor / handoff / network topologies; see per-agent context isolation; and judge when multi-agent actually helps.Advanced16m
    7. Context EngineeringCurate a finite context window: watch the budget fill as the loop runs, break down what fills it, apply the select/compress/isolate/persist levers, and avoid context rot.Advanced18m
    8. Loop EngineeringEngineer the agent control loop for reliability: gather → act → verify with retry-on-failure, bound it with budgets and loop detection, and add production reliability patterns.Advanced18m
    9. Fine-Tuning & LoRAAdapt foundation models to your task without retraining billions of parameters — explore the LoRA rank decomposition, compare full fine-tuning vs. PEFT strategies, and know exactly when to fine-tune vs. prompt or RAG.Advanced22m
    10. Quantization & Memory MathWork out whether a model actually fits. Split a float into range and precision bits, estimate inference at N × M × 1.2, and watch a 13B model need another 78 GB the moment you make every parameter trainable.Advanced20m
    11. Model MergingCombine models by mixing their parameters rather than their outputs. Build a task vector, use task arithmetic to add a capability or subtract one, interpolate along a sphere instead of a straight line, and see why TIES keeps only the top 20%.Advanced19m
    12. RLHF & AlignmentWalk through the full alignment pipeline — supervised fine-tuning, reward model training, and PPO — then see how DPO simplifies this by removing the RL loop entirely, and explore the failure modes that can make aligned models misbehave.Advanced24m
    13. AI AcceleratorsFind out which ceiling you are actually under. Walk the roofline from prefill to decode, climb the three-level memory hierarchy, and compute a real model's bandwidth utilization to see why quantization speeds up generation without doing less arithmetic.Advanced20m
    14. LLM Inference OptimizationUnderstand how production LLM serving achieves low latency and high throughput — explore KV cache mechanics, continuous batching, quantization tradeoffs, and speculative decoding side by side.Advanced22m
    15. KV Cache & Attention OptimizationWatch the KV cache turn quadratic recomputation into linear, then work the 2 × B × S × L × H × M formula that makes it 54 GB for a single Llama 2 13B batch — and see how Character.AI stacked three techniques to cut theirs by over twenty times.Advanced21m
    16. Speculative DecodingLet a small draft model guess K tokens ahead and have the real model check them all in one pass. Drive the accept-reject loop, see why the verification is close to free, and reproduce DeepMind's Chinchilla-70B result — half the latency, identical output.Advanced19m
    17. Eval Pipeline & GuardrailsGate a deploy on an offline eval suite, block injection & PII at runtime with guardrails, follow the dev→prod→feedback lifecycle flywheel, and learn what to observe in production.Advanced18m
    18. LLM Observability (OTEL / Langfuse)Explore the standard trace hierarchy — session → trace → span → generation — read OTEL GenAI semantic conventions, and debug a real failing session by clicking through spans, tool errors, and feedback.Advanced22m
    19. Prompt Injection & LLM SecurityTrace how untrusted content in a doc, page, or email hijacks an agent with tools — then stack defense-in-depth layers and see which attacks each one actually blocks.Advanced20m

    Learn it as part of a path

    AI Engineer: Zero to HeroMath & ML foundations → deep learning → the full LLM application pipeline → production AI systems.

    Related topics

    LLMs and transformersDeep LearningML Systems Design