Clarification on Bayesian ensembling I was reading a paper https://arxiv.org/pdf/2007.06823.pdf and at the end of page 3 the author presents the technique called "ensembling" for the estimation of the expected outputs and the associated uncertainties. In particular he writes

My question may be silly but i would like to have a confirmation: when he sums over the parameter theta_i, does that mean that the model is trained multiple time (one new training for each value of theta_i) or, since dropout randomly drop weights, it just means using multiple predictions y_i of the same value x_i obtained with the same trained model ? thanks

The model is not trained multiple times. The main idea is to model the weights as distribution instead of point wise (i.e. instead of a single number for each weight, each weight will be represented as a distribution). Usually, a natural pick for neural network, is to model the weights as a normal distribution, where each weight is represented by two numbers $\mu, \sigma$.
Now, you can sample from this distribution, and get a different result for each sample, and then you can average them.
The hard part is to prove that the optimization part is approximating to the posterior distribution (which is intractable this is why we only approximating), and this is done by bayesian inference.
