Position · last reconsidered September 17, 2026
A measure can break in four ways
Garrabrant argues that a number stops tracking what it was meant to track in at least four different ways once people chase it, and each needs its own fix.
- Areas
- KnowingAppearances
In our words
Scott Garrabrant takes Goodhart's lawGoodhart's law — When a measure becomes a target, it stops being a good measure: people learn to improve the number rather than what the number stood for., the observation that a measure stops working once people aim at it, and splits it into at least four failures. Picking the top scorers also picks the lucky ones. People raise the number by means that do nothing for the real goal. Push a number far enough and it leaves the range where it once matched the goal. Or someone games it on purpose, like a salesperson booking fake orders to hit a target. Each failure needs a different fix, so lumping them together hides what to do.
Scott GarrabrantRationalist community (LessWrong)2017
In their words
“However, this is not a single phenomenon. I propose that there are (at least) four different mechanisms through which proxy measures break when you optimize for them.”Connected ideas
derived from
- Goodhart TaxonomySource
The quoted passage and the argument around it (opening paragraph).
extends
- A measure that becomes a target stops measuringPosition
It turns Strathern's one sentence into a taxonomy with separate remedies.
related to
- ProxyConcept
The whole post is about how proxies break.
responds to
- What happens when the number becomes the goal?Question
It splits the question's single failure into four mechanisms.
- When does making something more efficient make it worse?Question
Each of the four types is a way optimization pressure destroys what a measure tracked.