Answers.Body: <blockquote>
    <p>Can the addition of dropout, in a non-overfitting neural network, increase accuracy?</p>
  </blockquote>
  
  <p>Yes, maybe.</p>
  
  <blockquote>
    <p>Even if I increase the complexity of the neural network?</p>
  </blockquote>
  
  <p>Yes, maybe.</p>
  
  <p>As always when making changes to ML algorithms, you need to test carefully to see if your changes have made an improvement. There are very few theories in non-linear machine learning models that make solid guarantees of results. One general difference you should note is that training a network with dropout will take longer (more epochs) than training a similar network without dropout, to reach the same levels of accuracy.</p>
  
  <p>However, as well as the regularisation effect of dropout, it shares some behaviour with ensemble techniques such as <a href="https://en.wikipedia.org/wiki/Bootstrap_aggregating" rel="nofollow noreferrer">bagging</a>. Dropout effectively trains many sub-networks (that share weights) on different samples of the training data. This pseudo-ensemble effect can boost accuracy, and other success metrics. This is not a guaranteed effect, but it does happen in practice.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-19T08:36:34.967Z
Answers.LastActivityDate: 2020-03-10T20:28:49.880Z
Answers.LastEditDate: 2020-03-10T20:28:49.880Z
Answers.UserDisplayName: Neil Slater
Body: <p>According to <a href="https://en.wikipedia.org/wiki/Dropout_(neural_networks)" rel="nofollow noreferrer">Wikipedia</a></p>
  
  <blockquote>
    <p>Dropout is a regularization technique for reducing overfitting in neural networks</p>
  </blockquote>
  
  <p>My neural network is simple enough and does not overfit.</p>
  
  <p>Can the addition of dropout, in a non-overfitting neural network, increase accuracy? Even if I increase the complexity of the neural network?</p>
  
Tags: <neural-networks><machine-learning><deep-learning><overfitting><dropout>
Title: Can the addition of dropout in a non-overfitting neural network increase accuracy?
UserDisplayName: Astariul
ThreadText: # Can the addition of dropout in a non-overfitting neural network increase accuracy?
  
  (Asked by: username_0 on 2018-10-19)
  
  According to [Wikipedia](https://en.wikipedia.org/wiki/Dropout_(neural_networks))
  
  > Dropout is a regularization technique for reducing overfitting in neural networks
  
  My neural network is simple enough and does not overfit.
  
  Can the addition of dropout, in a non-overfitting neural network, increase accuracy? Even if I increase the complexity of the neural network?
  
  **Tags:** neural-networks, machine-learning, deep-learning, overfitting, dropout
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-10-19. Score: 1 | Accepted answer
  
  > Can the addition of dropout, in a non-overfitting neural network, increase accuracy?
  
  Yes, maybe.
  
  > Even if I increase the complexity of the neural network?
  
  Yes, maybe.
  
  As always when making changes to ML algorithms, you need to test carefully to see if your changes have made an improvement. There are very few theories in non-linear machine learning models that make solid guarantees of results. One general difference you should note is that training a network with dropout will take longer (more epochs) than training a similar network without dropout, to reach the same levels of accuracy.
  
  However, as well as the regularisation effect of dropout, it shares some behaviour with ensemble techniques such as [bagging](https://en.wikipedia.org/wiki/Bootstrap_aggregating). Dropout effectively trains many sub-networks (that share weights) on different samples of the training data. This pseudo-ensemble effect can boost accuracy, and other success metrics. This is not a guaranteed effect, but it does happen in practice.
  
  
  ---
  
  