How to test if two events with unknown probabilities are different or not Let's say I have an "unfair die" with hundreds of possible results, and each result may have a different probability. However, I am interested only in two of these results and I want to test if they have the same probability or not.
Real data: After (at least) 500 rolls, the result A appeared 10 times and the result B appeared 44 times. How should I test this? And how should I involve the sample size? (I did at least 500 rolls, but they might even be 2000).
I've read about testing the fairness of a die using chi-square, but I don't know how to handle there the expected values, especially in relation to the number of times where the result was neither A or B.

It turns out the counts of the other outcomes don't matter: a suitable chi-squared test works just fine here.
Let the chance of outcome $A$ be $p.$ Under your null hypothesis, the chance of $B$ is the same and (therefore) the chance of seeing something other than $A$ and $B$ is $1-2p.$  Because your rolls are independent, probabilities multiply, implying the chance of what you observed is proportional to
$$L(p;10,44,500) = p^{10}p^{44}(1-2p)^{500-10-44} = p^{54}(1-2p)^{446}.$$
Upon taking logarithms and differentiating, it's easy to establish that this likelihood $L$ is uniquely maximized at the value $$p = \frac{1}{2} \frac{10 + 44}{500} = \frac{27}{500}.$$  The figure shows a plot of $L(p);$ the vertical axis is on a logarithmic scale.

Clearly, the expected counts under the null hypothesis are $27$ for both $A$ and $B$ and $500-2\times 27 = 446$ for all the others.  Because the expected count for all the others is equal to the actual count, it contributes nothing to the chi-squared statistic:
$$\chi^2 = \frac{(10-27)^2}{27} + \frac{(44-27)^2}{27} + \frac{(446-446)^2}{446} = 2\frac{17^2}{27} \approx 21.4.$$
(This is identical to the value you would obtain if you ignored all the non-A, non-B outcomes and just worked with the counts of $10$ and $44,$ exactly as if you were flipping a potentially unfair coin and the outcomes A and B corresponded to its two sides: for a fair coin, you would guess that both counts should be around $(10+44)/2=27$ and thereby obtain the same value of chi-squared.)
A large value of chi-squared indicates a large deviation from what would be predicted by the null hypothesis.  The chance of observing a statistic with a deviation at least this great is given by a chi-squared distribution.  The particular one to use has one "degree of freedom" because one unknown value, $p,$ was involved in the likelihood $L.$  The "p value" given by this distribution is less than four in a million, which is so small you can safely conclude the null hypothesis is incorrect.  In a technical paper you might write something like "the difference is significant ($\chi^2(1) = 21.4,$ $p = 4\times 10^{-6}$)."
Moreover, the evidence points to event $A$ having a smaller probability than $B.$
Finally, now that we know the two probabilities differ, we may estimate them (using maximum likelihood, as above, or otherwise) from the data.  The maximum likelihood estimates are $10/500$ and $44/500,$ respectively.
Reference
In a post at https://stats.stackexchange.com/a/17148/919 I give the background and necessary conditions for applying a chi-squared test.  You can verify all those conditions are satisfied here.
If the reasoning behind a null hypothesis and a p-value is unfamiliar, see my post at https://stats.stackexchange.com/a/130772/919 for an accessible account of this theory.
