Machine Learning, Deep Learning, Artificial Intelligence, AI, Neural Networks.
hessian matrix in that direction (eigenvector) and α is the regularized hyperparameter. Therefore, If λi ≫ α, the cost function is very sensitive in that direction and the corresponding weight reduces the cost significantly ⟹ don’t decay (shrink) much. If λi≪ α, the cost function is not sensitive