Linear Regression, Ridge Regression, Penalty, Loss, Cost Function.
does not depend on w. · Differentiating -2y^TXw with respect to w gives us -2y^TX. · Differentiating w^TX^TXw with respect to w requires the chain rule. The derivative is (X^TX + (X^TX)^T)w = 2X^TXw (since X^TX is symmetric). Differentiate the second term alpha * ||w||² with respect to w. Here,