Nassim Taleb: We should retire the notion of standard deviation
So first about the article:<p>>>The notion of standard deviation has confused hordes of scientists<p>What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ?<p>>>There is no scientific reason to use it in statistical investigations in the age of the computer<p>As someone who uses it daily I am eagerly awaiting his argument.<p>>>Say someone just asked you to measure the "average daily variations" for the temperature of your town (or for the stock price of a company, or the blood pressure of your uncle) over the past five days. The five changes are: (-23, 7, -3, 20, -1). How do you do it?<p>Ok... if I am to calculate the average I am calculating the average if I need to know standard deviation I calculate standard deviation...<p>>> It corresponds to "real life" much better than the first—and to reality.<p>What the flying fuck. What "real life" ? Standard deviation tells you how volatile measurements are not what mean deviation is. Those are both very real life things just not the same thing.<p>>>It is all due to a historical accident: in 1893, the great Karl Pearson introduced the term "standard deviation" for what had been known as "root mean square error". The confusion started then: people thought it meant mean deviation.<p>I don't know how one can read it and not think: "is this guy high or just stupid?".<p>>>. The confusion started then: people thought it meant mean deviation.<p>I am yet to see anybody who thinks that standard deviation is mean deviation. It's Taleb though. Baseless assertions insulting groups of people are his craft.<p>>>What is worse, Goldstein and I found that a high number of data scientists (many with PhDs) also get confused in real life.<p>One example please ?
I can give hundreds when std dev is useful and mean deviation isn't. Anything when you decide what % of yoru bankroll to bet on perceived edge for example.<p>Ok so he asserted that people should just use mean deviation instead of mean of squares. Guess what though, taking the squares have a purpose: it penalizes big deviations so two situations which have the same mean deviation but one is more stable have different standard deviations. THis information is useful for many things: risk estimation or calculating sample size needed for required confidence (if you need more experiments, how careful should you be with conclusions and predictions etc).
He didn't mention how are we going to achieve those with his proposal. Meanwhile he managed to throw insults towards various groups without giving one single example of misuse he describes.<p>This is not the first time he writes something this way. His whole recent book is like that. It's anti-intellectual bullshit with many words and zero points. He doesn't give any arguments, he throws a lot of insults, he misues words and makes up redundant terms which he then struggles to define.
The guy is a vile idiot of the worst kind: ignorant and aggressive. Him gaining so much following by spewing nonsense like this article is for sure fascinating but there is no place for him in any serious debate.