I Got More Data, My Model Is More Refined, but My Estimator Is Getting Worse [pdf]
I am the OP. For what it&#x27;s worth, I barely understand the paper, way too technical for me, so let me give some context as to why I posted this.<p>It was refered to in a course I&#x27;ve just started self learning, Berkeley&#x27;s Principles and Techniques of Data Science (aka Data 100), and the lecturer wanted to highlight the crucial importance of data acquisition in the Data Science lifecycle, sampling in particular, given that mistakes at this stage will propagate to the later stages. My understanding of his example (~1h10 into <a href="https:&#x2F;&#x2F;youtu.be&#x2F;JtwBwogRZkI" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;JtwBwogRZkI</a>) is that a 400 sized probability sample (a small random sample from the whole population) is often better than a millions sized administrative sample (of the kind you can download from gov sites). The reason is that an arbitrary sample (as opposed to a random one) is very likely to be biased, and, if large enough, a confidence interval (which actually doesn&#x27;t really make sense except for probability samples) will be so narrow that, because of the bias, it will actually rarely, if ever, include the true value we are trying to estimate. On the other hand, the small, random sample will be very likely to include the true value in its (wider) confidence interval.<p>The paper seems to elaborate on this, which I find interesting, and I was hoping for someone to explain it [EDIT: the paper, that is, I think I understand the lecture].