Which predictive models output the posterior distribution? In a supervised learning context, the posterior distribution of the target given the predictors is often discussed in foundational treatments of the subject. One way this comes up is in decision theory, which makes use of the said posterior distribution to produce the prediction, given some cost function.
In practice, which learning models actually output the posterior distribution? It appears popular decision-tree models and their implementations, such as XGBoost, and neural-network models do not readily output this distribution.

Only the Bayesian models, that are defined in terms of probability distributions, do. This is by definition, since posterior is obtained using Bayes theorem (see bayesian). Neither of the popular machine learning models does. Moreover, even if there are some Bayesian models implemented in the machine learning software (e.g. BayesianRidge regression in scikit-learn, usually they only produce maximum a posteriori estimates, i.e. find only the mode of the distribution, not the full posterior. For finding posterior distribution, in most cases you would need to implement the models by yourself in probabilistic programming languages like Stan, PyMC, TensorFlow Probability, Pyro, etc. Some of the models would be available out-of-the-box in packages such as R’s brms.
