Why is it good for inputs to a NN to be uncorrelated? According to the paper https://arxiv.org/pdf/1502.03167.pdf,

It has been long known (LeCun et al., 1998b; Wiesler & Ney, 2011) that
the network training converges faster if its inputs are whitened –
i.e., linearly transformed to have zero means and unit variances, and decorrelated.

My question is why would a network learn better from uncorrelated inputs?
My intuition for this is that if your inputs (X,Y) are independent (and in particular uncorrelated), you would expect the same conditional distribution of X no matter the value of Y, thus the data is in some sense more regular. But this is a very handwavy intuition and I would like to understand this claim on a deeper level.

Basically, gradient descent works best when the error surface is spherical - the same curvature (ie gradient of gradient) in each direction.
This is because you want to adjust the step size according to the curvature. Imagine in 1-d, if you have very strong curvature you need to take very small steps (or you will overshoot). If you have very low curvature you need a very large step size, or you don't get anywhere.  Now in more dimensions, different directions will have different curvature, and you will be forced to use the smallest step size across all your directions or you will overshoot the minimum, but that will mean you traverse other directions very slowly.
Now, if you consider the error surface of linear regression (ie take the 2nd derivative of the mean squared error wrt the weights(coefficients), you will  see that it is actually $X^TX$.  Whitening the inputs ($X$) will therefore give you a nice, spherical, bowl shape.  So to the extent that your problem has a linear component, you might expect this behaviour to carry over to your nonlinear problem.
see eg https://www.cs.toronto.edu/~rgrosse/courses/csc421_2019/slides/lec07.pdf around slide 19
