Common Probability Distributions: The Data Scientist’s Crib Sheet
Nicely done.<p>There is an important point that the article makes although only implicitly:  If have some data and want to know what the probability distribution is, then hopefully know enough good things about where the data came from basically to <i>know</i>, even without looking at the data, what the probability distribution <i>must</i> be.  The article gave such ways to know.<p>A biggie point:  In practice this way of <i>knowing</i> is not only powerful but, really, nearly the only little platform you have to stand on to know how your data is distributed.<p>Here is an example one step beyond the article:  You have a Web site, and users arrive.  Okay, each user has their own complicated life, maybe use the Internet only in the morning, only in the evening, have nearly a fixed list of sites they go to, only get to your site from links at other sites they do see regularly, etc.  That is, each user can have wildly complicated, unique personal behaviors on the internet.<p>Still, the arrivals at your site will be as in a Poisson process, that is, the number of arrivals in the next minute will have the Poisson distribution and the time until the next arrival will have the exponential distribution.  Why?  A classic result called the <i>renewal theorem</i>.  There is a careful proof in the second volume (the one difficult to read) on probability by W. Feller.<p>So, the arrivals at your Web site from user user #1, Joe, is some complicated, unknowable <i>stochastic arrival process</i>.  Fine.  Joe has a complicated life.  User #2, Mary, also has a complicated life but has essentially nothing to do with Joe (Joe is a nerd, and Mary is
pretty!).  So, Mary acts <i>independently</i> of Joe.  Similarly for users #2, 3, ..., 2 billion.  Then the arrivals at your Web site are the sum of those 2 billion complicated, unique, with details unknowable, independent arrival processes.  Then, with a few more meager assumptions, presto, bingo, the renewal theorem says that the arrivals at your site form a Poisson arrival process.<p>There&#x27;s a terrific chapter on the Poisson process in E. Cinlar&#x27;s introduction to stochastic processes.  Terrific.  Some of what you can say, knowing that you have a Poisson process, is amazing.  All with no or meager attention to the data and, instead, from knowing you have a Poisson process, e.g., from the renewal theorem from a sum of many independent arrival processes.<p>Bigger lesson:  The renewal theorem is true in the limit of a sum of many independent arrival processes.  So, it is a <i>limit theorem</i>.  Then, more generally, many of the crown jewels of probability are limit theorems that say what happens in the <i>big picture</i> when it is a limit of some kind of smaller things about which have nearly no ability to understand.  So, astoundingly, such limit theorems show that the effects of some universe of detail, maybe even <i>big data</i>, just wash out.  Often very powerful stuff.  A big part of a good course in probability is the full collection of classic limit theorems -- astounding, powerful stuff in there.  Wait until discover martingales -- totally mind blowing that any such powerful things could be true, but they are!<p>Final lesson:  It&#x27;s possible also to take from the article and from much of introductory lessons in statistics an implicit lesson that is wrong and even dangerous:  That lesson is that, given some data, right away, ASAP, do not pass GO, do not collect $100, and ASAP rush to find the probability distribution.  Well, if can find the distribution via something like the Poisson process outlined above, <i>terrific</i>.  But usually can&#x27;t do that.  Instead just have the data, just the darned data.  Maybe even <i>big data</i>.  Then, sure, can get a histogram and look at it.  Okay, no harm done so far.  But, then, maybe, from the implicit but dangerous lesson, feel an urge, a need, a compulsion, a strong drive to find <i>the distribution</i> of that data, go through some huge list of increasingly bizarre well known probability distributions looking for a <i>fit</i>, etc.  Mostly, don&#x27;t do that.<p>Or, yes, there is a probability distribution, but, usually in practice, especially when you are given data without any additional information that will let you conclude something like Poisson above, beyond just that histogram, you don&#x27;t have much chance of finding or approximating the probability distribution in any way that stands to be useful.  Or, mostly just get the histogram and stop there.<p>Next, all the above holds for one dimensional data, that is, single numbers.  But if your data comes in pairs of numbers, say, points on a plane, or, for some positive integer <i>n</i>, <i>n</i>-tuples, then your desire to find <i>the distribution</i> is much, much less promising.  Indeed, just getting a histogram is much less promising.  For <i>n</i> &gt; 2, already histograms are tough to see or work with.<p>But, fear not:  The field of applied probability and statistics is just awash in techniques where you don&#x27;t need anything like precise data on distributions!<p>Succinct version of this lesson:  Yes, the probability distribution exists, but commonly you can&#x27;t really find it and commonly you don&#x27;t need to find it.