algorithms.live
    ML-SYSTEMS
    beginner

    Framing ML Problems

    ML Engineer path · lesson 62 of 80
    The idea
    Two ways to ask a shop assistant what you should buy. You can ask 'which of these 40,000 items is for me?' — and every time the shop gets new stock, the assistant has to relearn the catalogue. Or you can hold up one item and ask 'is this for me?', forty thousand times. Same information, and only one of them survives a delivery.
    How to use
    On the Two framings tab, toggle the reframe and watch the output size and the cost of adding an app swap places.
    The sentence to start from
    “Slow customer support is a problem, but it's not an ML problem.”
    The bank example, worked
    the business problemcustomer support is slow→where the time actually goesrouting requests among four departmentsthe ML probleminputthe customer requestoutputthe departmentobjectiveminimise predicted vs actualaccounting · inventory · HR · IT — a four-class classification problem
    What the middle step did
    It found the bottleneck. “Support is slow” has no inputs and no outputs; “routing takes too long” has both. Most of the work in framing an ML problem is this move — locating the specific decision inside a vague complaint, and checking that a decision is what is actually slow.
    The objective function
    Also called a loss function, and distinct from both the business objective and the ML objective — this one is a mathematical function. The usual choices: RMSE or MAE for regression, logistic loss (log loss) for binary classification, cross entropy for multiclass classification.
    Cross entropy, worked
    ground truth [0, 0, 0, 1]  ·  prediction [0.45, 0.2, 0.02, 0.33]
    loss = −Σ p[i] · log(q[i])
    The model put its highest probability on tech and the truth was politics — so the loss is driven entirely by the 0.33 it assigned to the right answer.

    Key insight

    Inputs, outputs, objective function. If you cannot write all three down, you do not have an ML problem yet — you have a business complaint, and the work in front of you is product work rather than modelling work.

    Frame the task

    Sort

    Sort each problem into the ML task type it is.

    Tap a piece, then tap where it goes — or drag it. Tap the tray to take one back.

    Regression

    Drop items here

    Binary classification

    Drop items here

    Multiclass

    Drop items here

    Multilabel

    Drop items here
    5 pieces left to place

    Test your knowledge

    20 questions
    Question 1 of 20
    Score: 0

    An ML problem is defined by inputs, outputs, and:

    The short version

    Two ways to ask a shop assistant what you should buy. You can ask "which of these 40,000 items is for me?" — and every time the shop gets new stock, the assistant has to relearn the catalogue. Or you can hold up one item and ask "is this for me?", forty thousand times. Same information, and only one of them survives a delivery.

    In more depth

    Framing ML Problems

    "Slow customer support is a problem, but it's not an ML problem."

    An ML problem is defined by inputs, outputs, and an objective function. The bank example: the bottleneck is routing requests among four departments (accounting, inventory, HR, IT) ⇒ a classification problem. Input = the customer request. Output = the department. Objective = minimise the difference between predicted and actual.

    Types of ML tasks

    Classification ↔ regression — each reframes as the other. House price becomes classification by quantizing into buckets; email classification becomes regression by outputting 0–1 with a threshold at 0.5.
    Binary vs multiclass — binary is easier; F1 and confusion matrices are far more intuitive with two classes. The book hedges honestly on why binary problems are so common.
    High cardinality — ML models typically need at least 100 examples per class, so 1,000 classes needs ≥ 100,000 examples. And rare classes are where data is scarcest.
    Hierarchical classification — classify into a large group, then a subgroup.

    Multiclass vs multilabel

    Four topics [tech, entertainment, finance, politics]. Multiclass "entertainment" = [0, 1, 0, 0]; multilabel "entertainment and finance" = [0, 1, 1, 0].

    Multilabel is the task type companies have the most problems with, for two reasons:

    1.Label annotation is harder — the label multiplicity problem: one annotator says two classes, another says one.
    2.Extracting predictions from raw probability is hard — given [0.45, 0.2, 0.02, 0.33], multiclass takes 0.45; multilabel might take the two highest (0.45, 0.33) or the three highest (0.45, 0.2, 0.33).

    Two framings of the same problem

    Predicting the next app a user opens:

    Naive multiclassRegression framing
    Inputuser + environment features+ the app's features
    Outputdistribution over all N appsone value 0–1
    Predictions per slot1, of size NN, each a number
    A new app costsa retrainnothing

    Objective functions

    Also called loss functions: RMSE or MAE for regression, log loss for binary classification, cross entropy for multiclass. Worked: ground truth [0,0,0,1], prediction [0.45, 0.2, 0.02, 0.33], loss = −Σ p·log(q).

    Decoupling objectives

    Newsfeed ranking: extreme posts get more engagement, so engagement-maximisation prioritises extreme content. Five objectives, and quality conflicts with engagement.

    One model, combined loss: α·quality_loss + β·engagement_loss. Systematic tuning is Pareto optimization. Every retune requires retraining.
    Two models, combined at ranking time: α·quality_score + β·engagement_score. Tweak α and β without retraining.

    Second reason to decouple: different objectives need different maintenance schedules. "Spamming techniques evolve much faster than the way post quality is perceived."

    Interview Questions

    Stripe
    Intercom
    Amazon
    Zendesk
    Meta
    Pinterest
    Amazon
    Spotify
    Google
    Apple
    Meta
    Samsung
    Meta
    Twitter
    TikTok
    LinkedIn

    References & Further Reading

    Stanford MLSys Seminar Episode 5 — Principles of Good Machine Learning Systems DesignYouTube · Stanford MLSys
    video
    Introduction to Machine Learning Problem FramingGoogle for Developers
    docs
    Machine Learning Systems DesignChip Huyen
    course
    All ML Systems Design topics

    Love this visualization?

    Sign up to track your progress, earn achievements, use the AI assistant, and access your personal learning dashboard.

    100% free • No credit card required