How to prove unbiasedness/consistency/normality of an estimator that doesn't have a closed form? My estimator looks like this:
$$
\hat\theta(X) = \arg\max_{\theta} \frac1N \sum_{n=1}^N f(x_n|\theta)
$$
Here, $f(x_n|\theta)$ is some arbitrary function: it's not a logarithm, and the sum is not a log-likelihood.
I don't think that there's a closed form expression for $\hat\theta(X)$, so I'm using numerical optimization (gradient descent, Newton's method) to find it. However, this doesn't seem to let me calculate the expectation $\mathbb{E}\hat\theta(X)$ to prove even unbiasedness of the estimator.
How does one theoretically (without simulations) prove that such an estimator is (un)biased, (in)consistent, (non-)normal and so on?

On a general basis, $\hat\theta(X)$ is biased, due to the fact that it is equivariant under reparameterisation, i.e., if$$\eta=h(\theta)$$ is another parameterisation of the model, with $h$ a bijection, then $$\hat\eta(X)=h(\hat\theta(X))$$ while unbiasedness does not carry under arbitrary transforms $h$.
Note also that most parameters (or parameterisations) do not allow for the existence of an unbiased estimator, see e.g. the $\eta=1/p$ example for the Binomial model.
