Quantifying the confidence that the most sampled outcome is the most probable outcome Let us say we have a cheater's six-sided die, which we can assume to be unfair with an unknown probability distribution. We want to know the most likely roll with this die, and so we roll it $N \gg 6$ times and find that the frequency of rolling a 5 is higher than the frequencies of other rolls. How can we quantify our confidence that 5 is actually the most probable roll, i.e. that the underlying probability distribution of the die is peaked at the outcome 5?
It seems clear that if we roll less than 6 times, it is pretty random, while if we roll it a thousand times, we can be pretty confident due to the law of large numbers. Furthermore, if it was actually an almost fair die, with only a 1% higher chance to roll 5 than other outcomes, we would need many more rolls to identify this unfairness than if there was a 50% chance of rolling a 5.
However, I have so far been unsuccessful in quantifying these statements. Is there some way to estimate our confidence of identifying the most probable roll in terms of the number of trial rolls we have made, and the relative frequencies they have resulted in?

We can use profile likelihood methods to construct a confidence interval for the maximum probability $\theta = \max_{j=1}^k p_k$.  Here $p_1, p_2, \dotsc, p_k$ represent the discrete distribution of dice rolls, where in your example $p_5$ is somewhat larger than the others.  The results of $n$ dice rolls is given by the random variable $X=(X_1, \dotsc, X_k)$ where in the dice example $k=6$. The likelihood function is then
$$ L(p) = p_1^{X_1} p_2^{X_2} \dotsm p_k^{X_k} $$
and the loglikelihood is
$$ \ell(p) =\sum_1^k x_j \log(p_j)  $$
The profile likelihood function for $\theta$ as defined above is
$$ \ell_P(\theta) = \max_{p \colon \max p_j = \theta} \ell(p) $$
With some simulated data we get the following profile log-likelihood function

where the horizontal lines can be used to read off confidence intervals with confidence levels 0.95, 0.99 respectively.
(It is too late now, I will add details tomorrow)
