Construction of statistics of a discrete distribution I have the following problem: we consider an i.i.d sample $\mathbf{X} = (X_1,...,X_n)$ of the discrete set $\{1,...,N\}$. An agent has to infer the probability distribution of $X_i$.
I wanted to use the methods I'm used to with continuous statistics, but I guess I'm doing something wrong at one point. I modeled the problem as follow: we want to infer $\theta \in \{\theta \in \mathbb R\mid\sum_{i=1}^N \theta_i\ = 1, \theta_i \geq 0 \}$. The likelihood function is given by $L(x_1,...,x_m ; \theta) = \prod_{i=1}^m P(x_i \mid \theta)$ which correspond, as we expect $\mathbf X$ to follow the law given by $\theta$, $P(x_i \mid \theta) = \theta_{x_i}$
However, I'm stuck with the usual methods (MLE, Bayesian estimators...). Because of the dimensionality of the problem, I'm struggling to find any optimum or to conduct my calculation to an end.
However, the intuition is that I should be able to find a good estimator: for example, one way would be to count each occurrence of $i\in\{1,...,N\}$ and divide by the number of observations we've had until now.
What am I doing wrong?
Further info (in response to the first comment):
I'm stuck on the calculation:

*

*for the MLE firstly because there are multiple possible combinations $\theta_i, i\in\{1,...,N\}$ in the likelihood function - and second because even in the simple cases where $(x_1,x_2,x_3,...,x_n) = (1,2,3,..,N)$ the optimisation problem of finding a maximum for $L$ is difficult (sine $\theta$ is a parameter that lives in the simplex $\{\theta \in \mathbb R\mid\sum_{i=1}^N \theta_i\ = 1, \theta_i \geq 0 \}$, and that the partial derivatives fails to give simple solutions).

*For the Bayes estimator method, the set of different parameters is still the simplex, and is still difficult for me to understand how to deal with the mix of the discrete aspect of $P(x_i|\theta)$ and the continuous aspect of the posterior...

*My data is i.i.d. as an hypothesis (it is more on the theoretical side that on a real dataset - the data will be generated for my simulations).


The problem you describe is inferring the parameters of a Categorical distribution.
For the maximum likelihood estimator, note that you can write the likelihood as
$$\mathcal L = \Pi_{i=1}^m P(x_i | \theta) =  \Pi_{i=1}^N \theta_i^{n_i} $$
Where $n_i$ is the number of occurrences of the $i$-th discrete value. The only slightly tricky part here is that you want to maximize it over the simplex, namely under the constraint $\sum \theta_i = 1$. But that can be easily done  using a Lagrange multiplier :  we want to find (taking the log of the likelihood and adding the constraint)
$$ \frac{\partial}{\partial \theta_i} \left( \sum_i n_i \log \theta_i - \lambda( \sum_i \theta_i - 1) \right) = 0 $$
which trivially gives
$$ \hat \theta_i = \frac{n_i}{\lambda} $$
and the constraint implies that $\lambda = \sum_i n_i$, so the MLE turns out to be exactly like your naïve expectation.
For a Bayesian analysis, the Dirichlet distribution is a conjugate prior of this model, which means that the posterior is also a Dirichlet distribution with parameters that are straightforward to find. You can find the details
in this Section of the Wikipedia article.
