MLE for categorical distribution Referring to the picture, for each sample i in n, why do i only take the lambda for 1 class out of all k class instead of taking the product for the lambda's for all classes?


A categorical random variable is such that has $k$ possible values that are exclusive. For example, say that there are three possible ways you could pick to go to work: bike, car, and public transportation. You would pick one of them on a given day with probabilities $\lambda_1, \lambda_2, \lambda_3$ respectively. The options are mutually exclusive, so probabilities are such that $\sum_{j=1}^3 \lambda_j = 1$. The probability mass function for the distribution of this random variable is
$$ \prod_{j=1}^3 \lambda_j^{\mathbf{1}(x_j = 1)}
$$
which is a smart way of saying that you pick an appropriate $\lambda_j$ and ignore the others.
For the likelihood function, it assumes that the observations are independent and identically distributed, so it is
$$
\prod_{i=1}^n p(x_i) = \prod_{i=1}^n \prod_{j=1}^3 \lambda_j^{\mathbf{1}(x_{ij} = 1)}
$$
There is no reason why you would multiply $\lambda_j$’s. We multiply the probabilities of independent events, while here we are talking about mutually exclusive ones.
