Gradient Descent, Machine Learning, Neural Networks, Optimization, Linear Regression.
overshoot the minimum and fail to converge. With a large learning rate, the algorithm takes big steps towards the minimum, but it may continuously overshoot, causing the cost function to increase rather than decrease. This can lead to divergence, where the algorithm fails to find the optimal