Goodhart's Law and how systems are shaped by the metrics you chase
There was this concept of objectives and counter objectives which was meant to combat Goodhart’s law. The idea was that you define what you want to measure and then define another metric which acts as regularization for the first. A good example is if you define a metric of WAU only, you might be motivated to bring in as many new users each week as possible. Setting a counter metric of retention makes you take into consideration that it’s important they stay.<p>I don’t remember who suggested this - I vaguely remember it was someone at a16z, but can’t find the original place I read it