Dependence in probabilistic machine learning From this slide: slides page 39, it says:
In probabilistic machine learning prediction time: compute the likelihood given parameters, each parameter configuration of which is weighted by the posterior:
$$
p(y|x, D)=\int p(y|\theta, x) p(\theta|D) d\theta
$$
where $x, y, \theta, D$ is input, output prediction, network parameters, dataset, respectively. I am trying to derive this formula in this way:
$$
p(y|x, D)=\int p(y, \theta| x,D) d\theta = \int p(y|\theta, x, D) p(\theta|x, D) d\theta=\int p(y|\theta, x) p(\theta|D) d\theta
$$
However, it depends on such assumption: $y$ is independent of $D$, so $p(y|\theta, x, D)=p(y|\theta, x)$. Since $y$ depends on $\theta$, $\theta$ depends on $D$, it seems that $y$ should also depends on $D$. What's wrong here?
In addition, we need $p(\theta|x, D)=p(\theta|D)$, that is $x$ is independent of $\theta$. Is it still correct in training time? It seems that $\theta$ depends on $D$, so should also depend on $x$?

The form with $p(y|\theta,x,D)$ is the correct version according to Bayes Rule. But, in the literature around these, typically we assume $p(y|\theta,x,D)=p(y|\theta,x)$ because $D$ is data useful in estimating the parameters of the model, which is $\theta$. However, if you already know the parameters of the model, you only need the input and the model parameters to calculate $y$'s distribution.
For the second term, same logic applies. We estimate $\theta$ from data, not the current value of $x$ at prediction time.
