Question Regarding RSS for Linear Regression For linear regression we have that,
$$RSS(\beta) = ||Y-X\beta||^2$$
and the gradient,
$$\nabla RSS(\beta) = \nabla (Y-X\beta)^T(Y-X\beta)$$
Why isn't it $(Y-X\beta)(Y-X\beta)$? Why is the first term transposed?

$Y$ and $X\beta$ are vectors, so $Y-X\beta$ is a vector, and vector multiplication is ambiguous. Does it mean the dot product? Does it mean the cross product (and what does the "cross product" mean in an arbitrary number of dimensions)? Does it mean to multiply coordinate-by-coordinate? The usual matrix multiplication does not work in such a case, as it involves an $n\times 1$ vector times an $n\times 1$ vector, so even defaulting to such an option does not work.
Consequently, $(Y-X\beta)(Y-X\beta)$ is not really a meaningful mathematical expression.
However, we know what we want the result of that multiplication to be: one real number. Consequently, we want the dot product, typically denoted as a transposed vector times itself in the usual sense of matrix multiplication, and then $(Y-X\beta)^T(Y-X\beta)$ makes perfect mathematical sense.
