For MSE equation does order of $y$ and $\hat{y}$ in the residual $(y-\hat{y})$ matter? So the equation for MSE is $\frac{1}{2N}\sum(y-\hat{y})^2$. If you switch the order as in $\frac{1}{2N}\sum(\hat{y} - y)^2$ does that affect anything? The only thing I think it potentially effects is when you're doing gradient descent you have to change the sign in front of the learning rate multiplied by the derivative.

No because $a^2 = [(-1)(-a)]^2 = [(-1)^2(-a)^2] = (-a)^2$.
For the gradient, you'd have $2(-a)\left[\frac{\partial}{\partial \theta}(-a) \right] = 2a \left[\frac{\partial}{\partial \theta} a \right]$ due to the chain rule.
