Looking at the code for torch.distributions.Normal 11, it seems that mean is a property that returns self.loc which is a broadcasted version of loc ( the mean you specify  for the distribution). this extra broadcasting makes m.mean a non-leaf tensor.
Such an extra operation doesn’t seem to take place inside MultivariateNormal 8, so you can optimize that i think.
You might be better off optimizing mu in the case that is giving you trouble.