Q:

A question on accuracy of hypothesis testing

If I am doing a hypothesis test with a sample of 100 from a population of 1,000 it would be more precise than doing a test with a sample of 100 from a population of 100,000.  How can I show this? Is it the thing that is called "power of the test"? Where can I read some easy-to-understand guides about it and how to calculate it for my hypothesis tests?
Thank you all in advance!

A:

No, the power of a given statistical test is a measure of how large chance you have of getting a significant result with a certain data set given a certain effect size. If you have a guess of what the effect size is, this can then be used to calculate how many observations you should make in order to have a reasonable chance of getting a useful result.
That 100 out of 1000 gives you a more accurate representation of the population than 100 out of 100000 isn't that strange. Let's assume that you have a population consisting of $X$ number of data points, where each data point can have a value between $a$ and $b$ (where $b>a$).
Now, if you sample $X$ number of data points, your sample mean will naturally be equal to the population mean. If you, on the other hand, sample $X-1$ data points, your sample estimate can theoretically not be more off than 
$\frac{(X-1)*a+b}{X}-\frac{(X-1)*a}{(X-1)}=\frac{(X-1)}{X}a+\frac{1}{X}*b-a$
from the population mean (that is, in the scenario where you happen to have $X-1$ data points with the value $a$, which you've sampled, and one data point with the value $b$, which you haven't sampled).
More generally, if you draw a sample of size $\delta$ (where $\delta \leq X)$, you can theoretically not be more off than
$\frac{\delta*a+(X-\delta)*b}{X}-\frac{\delta*a}{\delta}=\frac{\delta}{X}*a+\frac{(X-\delta)}{X}*b-a$
Now, as $\delta$ goes from $1$ towards $X$ (that is, as the sample size increases), we can see that the weight of $a$ increases while the weight of $b$ decreases, meaning that the whole expression decreases since $b>a$. (Remember that the two weights, $\frac{\delta}{X}$ and $\frac{X-\delta}{X}$, always has the sum 1 when put together.) This means that as the sample size gets bigger, the theoretical possibility off how much the sample mean can differ from the population mean shrinks. 

