algorithms.live
    AI-ENGINEERING
    intermediate

    Instruction-Following Evaluation

    AI Engineer path · lesson 90 of 124
    The idea
    A competent chef handed a recipe card. Whether the food is good is one question; whether they made the dish you asked for, in the portion size you specified, without the ingredient you said you were allergic to, is a completely different one. The second question is the one your program depends on.
    How to use
    On the IFEval tab, step the five instruction groups — every one of them is checkable by a script, which is the whole design constraint.
    Instruction following is one of the four buckets of evaluation criteria, alongside domain-specific capability, generation capability, and cost and latency. It is the one that decides whether your code can use the answer.
    The same model, two different failures
    Domain failure
    Asked for the sentiment of an angry tweet, the model answers POSITIVE. The format is right and the understanding is wrong.
    Instruction-following failure
    Asked for one of NEGATIVE, POSITIVE or NEUTRAL, the model answers ANGRY. The understanding is correct and the response is unusable.
    InstructGPT was named for being finetuned to follow instructions, and more powerful models generally follow instructions better — GPT-4 over GPT-3.5, Claude-v2 over Claude-v1. But it does not come for free with capability, which is why it is measured separately.
    It goes well beyond structured output
    “Use only words of at most four characters” is an instruction, not a format. Ello, a startup helping children read better, needs a model that can generate stories from a limited pool of words the child already knows — a constraint no schema can express and no parser can repair.
    Why it is hard to measure
    The capability is confounded with every other one. If a model fails to write a lục bát — a Vietnamese verse form — that could mean it does not know the form, or that it did not understand the instruction, and the output looks the same either way.
    “When a model performs poorly, it can either be because the model is bad or the instruction is bad.”
    Both benchmarks below are answers to that problem. IFEval sidesteps it by only using instructions whose satisfaction is mechanically checkable. INFOBench accepts it and decomposes each instruction into criteria a judge can rule on one at a time.

    Key insight

    This is the capability that decides whether an AI feature can be a component rather than a chat window. A model that is brilliant and ignores your output contract cannot be called by a program — and the fix is rarely a better model, it is a measurable constraint plus one of the enforcement layers.

    Can a script check this?

    Sort

    Sort each instruction by whether compliance is verifiable in code.

    Tap a piece, then tap where it goes — or drag it. Tap the tray to take one back.

    Verifiable by code

    Drop items here

    Needs a judge

    Drop items here
    6 pieces left to place

    Test your knowledge

    20 questions
    Question 1 of 20
    Score: 0

    A model correctly identifies an angry tweet but answers "ANGRY" when asked for NEGATIVE, POSITIVE or NEUTRAL. This is a failure of:

    The short version

    A competent chef handed a recipe card. Whether the food is good is one question; whether they made the dish you asked for, in the portion size you specified, without the ingredient you said you were allergic to, is a completely different one. The second question is the one your program depends on.

    In more depth

    Instruction-Following Evaluation

    One of the four buckets of evaluation criteria — domain-specific capability, generation capability, instruction-following capability, and cost and latency.

    The distinction

    A model can understand tweet sentiment (domain capability) and still output HAPPY/ANGRY instead of the requested NEGATIVE/POSITIVE/NEUTRAL (instruction following). It goes beyond structured output: "use only words of at most four characters" is an instruction no schema expresses. Ello needs stories generated from a limited pool of words a child already knows.

    The measurement problem: it is confounded with every other capability. Failing to write a lục bát could mean the model doesn't know the form or didn't understand the instruction.

    "When a model performs poorly, it can either be because the model is bad or the instruction is bad."

    IFEval (Google, Zhou et al., 2023)

    25 types of automatically verifiable instruction. Score = fraction of instructions followed correctly. Five groups:

    Keywords — include keywords, keyword frequency, forbidden words, letter frequency
    Language — response language
    Length constraints — paragraphs separated by ***, number of words, number of sentences, paragraphs + first word of the i-th paragraph
    Detectable content — postscript, number of [placeholder]s
    Detectable format — number of bullets, title in <<double angular brackets>>, choose from options, minimum *highlighted sections*, section splitters, JSON format

    Narrow by design, and in exchange objective, reproducible and free to compute. Added to the Hugging Face leaderboard in the June 2024 revamp.

    INFOBench (Qin et al., 2024)

    Broader — also content constraints ("discuss only climate change"), linguistic guidelines ("use Victorian English") and style rules. Verification uses a list of yes/no criteria per instruction.

    Worked example — "Make a questionnaire to help hotel guests write hotel reviews": (1) Is it a questionnaire? (2) Is it designed for hotel guests? (3) Is it helpful for writing hotel reviews? A model succeeds only if it meets all criteria; partial credit is 2/3 if two of three are met. Final score = criteria met ÷ total criteria.

    GPT-4 was found to be a reasonably reliable and cost-effective evaluator — not as accurate as human experts, but more accurate than Amazon Mechanical Turk annotators.

    Roleplaying

    Two purposes: a character for users to interact with, and a prompt engineering technique. An LMSYS analysis of one million Vicuna demo and Chatbot Arena conversations found roleplaying is the eighth most common use case.

    Evaluate on both style and knowledge — a Jackie Chan bot should capture his style and be limited to his knowledge. "Negative knowledge" checks matter for gaming: an NPC shouldn't accidentally give players spoilers. Benchmarks: RoleLLM (similarity + AI judges), CharacterEval (human annotators + a trained reward model, five-point scale).

    Curate your own

    If you need YAML, test YAML. If you don't want "As a language model", test for it.

    Interview Questions

    OpenAI
    Google
    Anthropic
    Meta
    Google
    Hugging Face
    Meta
    Google
    Anthropic
    Scale AI
    OpenAI
    Anthropic
    Microsoft

    References & Further Reading

    AI Engineering in 76 Minutes — Evaluating AI ModelsYouTube · Marina Wyss
    video
    Instruction-Following Evaluation for Large Language Models (IFEval)arXiv
    paper
    InFoBench: Evaluating Instruction Following Ability in Large Language ModelsarXiv
    paper
    phoenixtype/ai-engineering-interview-prepGitHub
    docs
    All AI Engineering topics

    Love this visualization?

    Sign up to track your progress, earn achievements, use the AI assistant, and access your personal learning dashboard.

    100% free • No credit card required