Clarification on the connection between deep ensembles and bayesian neural networks I'm not sure if I understand the relationship between deep ensembles and Bayesian interpretation well. can you tell me if i am right or wrong?
Suppose I have an ensemble of L neural networks trained through the maximum likelihood principle to predict for each point $x_i$ the mean $\mu(x)$ and the variance $\sigma(x)$  of a Gaussian distribution ( i am assuming that each point $y$ is independent and gaussian distributed around the mean $\bar{y}_i$ with a variance $\sigma_{\bar{y}_i}$). I have read that an ensemble of neural networks  approximate the posterior distribution of the weights as a mixture of delta functions centered around  certain configurations $w^l$ of the weights which correspond to MAP solutions
$$   P(\bf{w}|\bf{X}_{tr},\bf{Y}_{tr}) = \frac{1}{L} \sum_{L}\delta(\bf{w}-\bf{w}^l)$$
Is it correct to say that under these conditions the predictive posterior distribution will end up being a mixture of Normal distributions?
$$        P(\bf{y}|\bf{x},\bf{X}_{tr},\bf{Y}_{tr}) = \int_{\mathcal{W}} P(\bf{y}|\bf{x},\bf{w}) P(\bf{w}|\bf{X}_{tr},\bf{Y}_{tr}) \,d\bf{w}   \approx \frac{1}{L}\sum_{l=1}^L \mathcal{N}(\bf{y};\hat{\bf{y}}_{\bf{w}_l}(\bf{x}),  \bf{\hat{\Sigma}}^2_{\bf{w}_l}(\bf{x}) )  $$

Bayesian neural network is a fake technology. Take any example, which shows distribution of the target and change data into multimodal, just split them into two blocks with the gap between, launch BNN and it returns you same unimodal distribution. BNN only returns correctly expectation and variance. But you can get same result by two deterministic models. Make one model for targets as $t = M_E(x)$. Introduce variance as feature dependent value ${v = [t - M_E(x)]}^2$ and build second deterministic model for variance as $v = M_v(x)$. Compare it to BNN result and you'll see that they match.
