Don’t Let Metrics Undermine Your Business
I work as a data scientist on an extremely data driven team at an extremely data driven company, and I couldn&#x27;t agree more strongly. Everyone I work with would agree too. &quot;Numbers aren&#x27;t everything!&quot; says the person whose job it is to deal with business problems via numbers.<p>But this advice is so hard to implement. It&#x27;s like the whole null hypothesis p-value debacle going on in academic publishing right now. Sure we can agree it&#x27;s no good, but there&#x27;s no <i>standard</i> alternative.<p>Standards are powerful. They break lots of ties. All else equal, if you have a hunch and I have a measurement tied to objective reality, people will side with me. Nobody ever got fired for choosing Java. Nobody ever got fired for doing the thing that made The Metrics go up.<p>It&#x27;s so reassuring to just say &quot;We still disagree, so let&#x27;s just test it.&quot; We&#x27;re submitting our dispute to adjudication via Science by resolving via The Metrics! It&#x27;s much harder to be objective without these kinds of numbers, and I like being objective! I want the scientific method to tell me I did the right thing and I&#x27;m a good person.<p>Because without that, when we&#x27;re trying to figure out what to do, it&#x27;s just us, you know? It feels like guessing, and that&#x27;s hard.