Class 7 · CBSE AI · Strand A — Systems Thinking
Goodhart's Law — when AI optimises the wrong thing
'When a measure becomes a target, it stops being a good measure.' Why AI chases the wrong goal. For Class 7.
What this concept actually says
- Metric substitution happens when a system optimises for a measurable proxy instead of the actual goal
- Goodhart's Law states: when a measure becomes a target, it ceases to be a good measure
- Identifying the gap between the proxy metric and the true goal is one of the most important skills in AI system design
An analogy your child will recognise
Board exam preparation in India
Every student in India knows the difference between 'studying for boards' and 'actually understanding the subject.' The board exam is the proxy metric — it's supposed to measure knowledge, but when it becomes the target, students memorise model answers and forget the concept the day after the exam. The metric rose; the goal didn't.
Call centre targets
A call centre sets a target of 'handle 40 calls per day per agent.' Agents start ending calls early to hit the number — customers' problems remain unsolved. The manager measures productivity; what actually got measured was the speed of getting customers off the phone. Goodhart's Law in every Indian office that has ever had call-time targets.
Common misconceptions to watch for
- If a metric is improving, the underlying goal must be improving too.
- Gaming a metric requires bad intentions — in reality, well-meaning actors game metrics simply by responding rationally to incentives.
Key facts in one breath
- Goodhart's Law: 'When a measure becomes a target, it ceases to be a good measure.' Coined by British economist Charles Goodhart.
- Metric substitution is choosing a measurable proxy (like clicks) instead of the true goal (like informed citizens) because the proxy is easier to track.
- Systems optimised on the wrong metric can appear to be performing well while causing serious harm.
- Adversarial users and the system itself can both 'game' a metric — designers must anticipate both.
How Dhee Learning teaches this — the 3-stage question loop
Every Dhee Learning session for this concept follows three stages. We share the questions Dhee actually asks, so you can hear what a session sounds like.
Stage 1 — Surface
If a school decides the best teachers are those whose students score highest on exams, and starts paying teachers based on exam results — what do you predict happens over time?
Rote answer
"Teachers might cheat or only teach exam topics."
Understood
"The metric (exam score) becomes the target, so teachers focus entirely on it. But the real goal — deep understanding and curiosity — isn't measured by the exam. So the metric rises while the actual goal erodes. The measure stopped being a good measure because it became the target."
Stage 2 — Reasoning
A hospital is rated by how quickly patients are discharged. The AI scheduling system is optimised to minimise average stay length. What is the real goal the hospital should be optimising for — and how might minimising stay length conflict with it?
Follow-up Dhee may use: Can you design a better metric — one that is still measurable but harder to 'game'? What would you measure instead?
Stage 3 — Application
Here is a list of AI systems and their optimisation metrics. For each one, identify the likely gap between the metric and the true goal, and name one way the system might 'game' the metric: (a) news app optimised for clicks, (b) fitness AI optimised for daily step count, (c) customer service chatbot optimised for call resolution rate.
Misconception Dhee watches for: Child assumes the metric and the goal are always aligned if the designers chose the metric in good faith.
Related concepts
Want your child to actually understand this?
Dhee turns this concept into a 15-minute spoken session — asking, listening, and probing — so your child builds the idea themselves.
Frequently asked questions
What is when a system optimises the wrong thing — explained for kids? +
'When a measure becomes a target, it stops being a good measure.' Why AI chases the wrong goal. For Class 7.
What's the most common mistake children make about this concept? +
If a metric is improving, the underlying goal must be improving too.
How does Dhee Learning teach this in a Class 7 session? +
Dhee opens with a question — for example: "If a school decides the best teachers are those whose students score highest on exams, and starts paying teachers based on exam results — what do you predict happens over time?" — listens to your child's answer, then probes the reasoning behind it. The session ends when the child can apply the idea to a brand-new situation, not just recall it.