Is it a good idea to use classification to approximate a probability distribution? I have a dataset of multiple continuous variables associated with a target value. I would like to estimate the probability distribution of the output given new data. (I have around 50-120 features with 2.5 million samples.)
In an MLP classification approach, I would bin the output into 1000 bins(classes) then train it with the data and obtain the probability for each class.
If I go this way, what hyperparameters (loss function, activation function, layers) are suitable for this task?
If an MLP classifier isn't a good idea, what other approaches can I use for probability distribution estimation?

You explain your objective better in a comment than in the body of the question:

For example if I have the data of turtle sizes with their speed, I would like to estimate the probability distribution of the speed of the turtle (probabilities of the turtle moving at different speed) given its size. Then I can apply the same model to a new turtle to find out its possible speeds given its size.

The turtle analogy is fun, so I take it at face value in my answer. In your actual application you have 50+ features which only exacerbates the problems with your approach.

Effectively you propose to discretize the conditional distribution of speed given size into a histogram. As @Dave points out, this results in a huge loss of information: 1) to cut a continuous variable into intervals (bins) assumes that there is no meaningful variability inside each bin and a discontinuous jump between bins; 2) to treat the histogram bins as classes ignores their natural ordering.
Not discussed in your proposal but implied nevertheless is the fact that you would have to discretize size as well. I assume that size is a continuous variable*, so it's unlikely that there are two or more turtles with exactly the same size in your data. Another round of information loss.
Choosing a complex model — Multi-layer Perceptron classifier — will not help you recover the information you lose from ironing out the details in your data.
You could estimate the distribution of speeds given weight with quantile regression, by fitting a model for a choice of quantiles, eg., {0.1, 0.25, 0.5, 0.75, 0.9}. The more quantiles, the finer approximation to the conditional distribution of speed given weight. Keep in mind that it is more difficult to estimate the tails of a (unimodal) distribution than its center. This means you'll need to observe a lot of turtles.
Or you could specify the form of $f_\theta(\text{speed} | \text{size})$ as a function of a few parameters $\theta$ and optimize.
The best approach, however, is to collect more data. There is unexplained variability in speed after conditioning of size. What additional information can help model this variability: turtle species? gender? temperature? terrain (if turtles are observed in the wild) or stimuli (if turtles are observed in a lab), ...
Once you have more features that are predictive of turtle speed, you can build a more sophisticated model (withou discretization) $\text{speed} \sim f(\text{size},x_2,\ldots,x_k)$. Then you can get an idea of the distribution of speeds given size by marginalizing out predictors $x_2,\ldots,x_k$.
٭ Instructions for Taking Turtle Measurements. Alabama Outdoor Classroom Box Turtle Research Program by the Alabama Wildlife Federation.
