Stochastic Gradient, Machine Learning, AI, Gradient Descent.
be biased towards simpler models on the basis that they are capturing something more fundamental. Regularisation is a concrete method for implementing the principle of Occams Razor. The idea is to add a penalty term. For the case of Linear Regression the new optimisation problem is to minimise: