AI & Machine LearningMachine Learning Evaluation

When a Measure Becomes a Target

Goodhart's law catches AI and people alike

Hanoi once paid a bounty for every rat tail, to get rid of rats. Soon people were raising rats. How?

You'll find out

  • How a rat bounty bred rats
  • Why a hit target can mean a missed goal
  • Why a top AI score can mislead

Why it matters

Grades, KPIs and AI leaderboards all face this. Knowing it helps you judge any claim built on one number.

About 3 min6 interactive slides · 5 questionsLesson 686 · Zone 77

Play this lesson

Free to play in your browser or on iPhone

More in this zone

  1. 685Thermal PhysicsCan Hot Water Freeze Faster?Scientists still argue about the Mpemba effect
  2. 687Decision TheoryNewcomb's ParadoxA predictor who's always right offers you two boxes
  3. 688Software HistoryY2KSaving two digits almost broke the year 2000
  4. 689Software HistoryThe 2038 ProblemMany computers' clocks will run out of numbers on January 19, 2038
  5. 690Hardware SecurityWhere Your Face Is StoredA locked chip inside your phone guards secrets from the phone itself
  6. 691Cryptocurrency HistoryThe Bitcoin PizzaIn 2010 someone paid 10,000 bitcoins for two pizzas
  7. 692Fluid MechanicsThe Shrimp That Shoots BubblesA collapsing bubble can be hot enough to glow
  8. 693Reinforcement LearningAI Plays Hide and SeekAgents found tool tricks and exploits nobody taught them