Estimation of Bernouilli parameter: what confidence interval? Two years ago I answered the question: What are the odds in Scrabble of not being able to make a legal move on opening?
This question is asking for the value of the parameter in a Bernouilli distribution. A player is dealt a hand of letter tiles; then either they can form a word, with probability $1-p$, or they cannot form a word, with probability $p$. We want to find $p$.
I wrote a quick program that worked as follow:

*

*Repeat 50 times:

*Draw 1000 hands of Scrabble, and count how many of those 1000 hands cannot form a word.

I end up with a list of 50 frequencies of failure: [4/1000, 2/1000, 8/1000, 4/1000, 5/1000, ...].
An obvious estimator of $p$ is the average of those 50 values. But what is the error margin around this value?
In my answer, I simply calculated the average 5.8/1000 and standard deviation 2.7/1000 of those fifty values, and concluded:

The result is a $0.58\% \pm 0.27\%$ probability that no word can be formed.

Later, someone came up with combinatorics arguments to reduce the search space and was able to run an exhaustive search, ending with the conclusion that the exact value of $p$ was $\frac{91,595,416}{16,007,560,800}$ or 0.5722%.
As you can see, my estimation of $p$ was pretty accurate: 0.58% instead of 0.5722%. However, my standard deviation was disappointingly huge; writing $p = 0.58\% \pm 0.27\%$ looks very imprecise.

*

*What better confidence interval could I have given? Is there a concentration inequality particularly well-suited for this setting?

*Was I correct to split my 50000 hands into 50 experiments of 1000 hands? What would be a better way to use these 50000 i.i.d Bernoulli trials?

It would make more sense to say you had $50000$ attempts and $290$ successes.  You have no reason to split it in the way you did, and by doing so you lost the useful information that each experiment had $1000$ attempts
This gives the same estimate of $\hat p = 0.0058$ or $0.58\%$
There are several possible ways of calculating a binomial proportion confidence interval, but you might end up with something like $[0.0051,0.0065]$ or $0.58\% \pm 0.07\%$ which is a lot narrower than your batching suggested
