Resampling small datasets - Issue of overcounting? Hypothetical situation here, as techniques like bootstrapping often fail for very small datasets.
Taking bootstrapping as an example nonetheless. we can easily compute the number of possible bootstrap (re)samples.
A nice answer is given by @whuber here:
Amount of Possible Bootstrap Samples
Suppose for a moment that bootstrapping is perfectly valid for low sample sizes. Now, say we have $n = 5$. From the posted solution above, it is found that there are a total of 126 possible boostrap resamples that can be drawn.
In bootstrapping, we usually take a large number of replications (10000 for instance). Taking this many replicates for such a small dataset, like the one above, seems bizarre, since resamples will be counted multiple times.
Question: Does this really matter? If yes. what is the effect on inference?

Many bootstrap replications are drawn in order to approximate (in a standard situation) the distribution of i.i.d. samples from the empirical distribution. Now if you can get this distribution explicitly, which is possible with a small sample as you can list all possible samples and their probabilities, it isn't necessary to approximate it by a much larger number of random bootstrap samples. Instead you can just use the full distribution of bootstrap samples (obviously potential problems with a lack of representativity of your small sample as mentioned in other answers still exist, but I believe this was not the question).
Note by the way that i.i.d. sampling from the empirical distribution will produce uniform probabilities over ordered rather than distinct samples, meaning that if you want to emulate the true bootstrap distribution by a set of samples, you will need some repetition of most of the 126 distinct samples. This would be approximated by randomly taking a very large number of bootstrap samples, so multiple counting of samples is not the issue here, rather what happens is you're using more computing power than necessary to do something less precise than possible with less computing effort.
