The Human Side of ML

The Human Side of ML

Three differences, three challenges

Challenge 1 — consistency

Booking.com has about 200 filters. "There are so many filters that it takes time for users to find the filters that they want." The failure: "if their ML model kept suggesting different filters each time, users could get confused, especially if they couldn't find a filter that they had already applied before."

The fix is a rule, not a better model: conditions where the system must return the same recommendations (the user has applied a filter) and conditions where it may return new ones (the user changes destination).

The consistency-accuracy trade-off: "the recommendations deemed most accurate by the system might not be the recommendations that can provide user consistency."

Challenge 2 — mostly correct

Here you want less consistency and more diversity. The irony that kills the naive version: "A React engineer might be able to fix this code quickly, but many users of this application might not know React. And this application might attract a lot of users who don't know React — that's why they needed this app in the first place!"

The fix: *"Show users multiple resulting predictions for the same input to increase the chance of at least one of them being correct. These predictions should be rendered in a way that even nonexpert users can evaluate them."* This is human-in-the-loop AI.

Challenge 3 — failing smoothly

A backup system: "less optimal than the main system but... guaranteed to generate predictions quickly." Heuristics, simple models, or cached precomputed predictions. The rule: "if the main model takes longer than X milliseconds to generate predictions, use the backup model instead." The sophisticated variant predicts the main model's latency and routes — with the caveat that that model also adds latency.

"A model might have worse performance than another model but can do inference much faster... Many companies have to choose one model over another, but with a backup system, you can do both."

Subject matter experts

Doctors, lawyers, bankers, farmers, stylists — "they're not only users but also developers of ML systems." The mistake: "most people only think of subject matter expertise during the data labeling phase." They belong in problem formulation, feature engineering, error analysis, model evaluation, reranking predictions, and user interface.

Two hard problems: explaining ML limitations to people without engineering backgrounds, and versioning domain expertise — "good luck trying to get your doctor to use Git." The answer is no-code and low-code platforms, today mostly at the labeling, quality assurance and feedback stages.

Team structure

Approach 1 — a separate team productionizes. Easier hiring; four drawbacks: coordination overhead (Brooks: "What one programmer can do in one month, two programmers can do in two months"), debugging, finger-pointing, narrow context.

Approach 2 — data scientists own everything. Without tools they become "grumpy unicorns." "In theory, you can learn both sets of skills. In practice, the more time you spend on one means the less time you spend on the other." Erik Bernhardsson: like expecting app developers to know how Linux kernels work.

"For data scientists to own the entire process, we need good tools. In other words, we need good infrastructure."

Netflix's full-cycle developer model: "the specialists — people who originally owned a part of the project — first create tools that automate their parts."