Answers.Body: <p>Also, keep in mind that not just any augmentation of the loss function is a regularization.</p>
  <p>For example, you can add terms to a loss function that enforce constraints on the solution but do not prevent overfitting nor facilitate generalization.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-07-27T18:47:46.990Z
Answers.LastActivityDate: 2021-07-27T18:47:46.990Z
Answers.UserDisplayName: ChalkTalk
Answers.Body: <p>Regularization is not limited to methods like L1/L2 regularization which are specific versions of what you showed.</p>
  <p>Regularization is any technique that would prevent network from overfitting and help network to be more generalizable to unseen data. Some other techniques are Dropout, Early Stopping, Data Augmentation, limiting the capacity of network by reducing number of trainable parameters.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-07-27T09:41:24.210Z
Answers.Text: L2 regularisation is an interesting case, in that there are two equivalent ways of implementing it. You can use an adjustment to the loss function, or you can decay the weights after each update step
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-07-27T08:08:24.743Z
Answers.LastActivityDate: 2021-07-27T08:08:24.743Z
Answers.UserDisplayName: meliksahturker
Body: <p>We need to use a loss function for training the neural networks.</p>
  <p>In general, the loss function depends only on the desired output <span class="math-container">$y$</span> and actual output <span class="math-container">$\hat{y}$</span> and is represented as <span class="math-container">$L(y, \hat{y})$</span>.</p>
  <p>As per my current understanding,</p>
  <blockquote>
  <p>Regularization is nothing but using a new loss function
  <span class="math-container">$L'(y,\hat{y})$</span> which must contain a <span class="math-container">$\lambda$</span> term (formally called
  as regularization term) for training a neural network and can be
  represented as</p>
  <p><span class="math-container">$$L'(y,\hat{y}) = L(y, \hat{y}) + \lambda \ell(.) $$</span></p>
  <p>where <span class="math-container">$\ell(.)$</span> is called regularization function. Based on the
  definition of function <span class="math-container">$\ell$</span> there can be different regularization
  methods.</p>
  </blockquote>
  <p>Is my current understanding complete? Or is there any other technique in machine learning that is also considered a regularization technique? If yes, where can I read about that regularization?</p>
  
Tags: <machine-learning><reference-request><terminology><regularization>
Title: Does regularization just mean using an augmented loss function?
UserDisplayName: hanugm
ThreadText: # Does regularization just mean using an augmented loss function?
  
  (Asked by: username_0 on 2021-07-27)
  
  We need to use a loss function for training the neural networks.
  
  In general, the loss function depends only on the desired output $y$ and actual output $\hat{y}$ and is represented as $L(y, \hat{y})$.
  
  As per my current understanding,
  
  > Regularization is nothing but using a new loss function
  > $L'(y,\hat{y})$ which must contain a $\lambda$ term (formally called
  > as regularization term) for training a neural network and can be
  > represented as
  >
  > $$L'(y,\hat{y}) = L(y, \hat{y}) + \lambda \ell(.) $$
  >
  > where $\ell(.)$ is called regularization function. Based on the
  > definition of function $\ell$ there can be different regularization
  > methods.
  
  Is my current understanding complete? Or is there any other technique in machine learning that is also considered a regularization technique? If yes, where can I read about that regularization?
  
  **Tags:** machine-learning, reference-request, terminology, regularization
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2021-07-27. Score: 3 | Accepted answer
  
  Regularization is not limited to methods like L1/L2 regularization which are specific versions of what you showed.
  
  Regularization is any technique that would prevent network from overfitting and help network to be more generalizable to unseen data. Some other techniques are Dropout, Early Stopping, Data Augmentation, limiting the capacity of network by reducing number of trainable parameters.
  
  > **Comments:**
  > **username_3**: L2 regularisation is an interesting case, in that there are two equivalent ways of implementing it. You can use an adjustment to the loss function, or you can decay the weights after each update step  
  
  ---
  
  ### Answer by username_1 on 2021-07-27. Score: 1
  
  Also, keep in mind that not just any augmentation of the loss function is a regularization.
  
  For example, you can add terms to a loss function that enforce constraints on the solution but do not prevent overfitting nor facilitate generalization.
  
  
  ---
  
  