kpatil:

I do not get the point why a dropout layer should not to be set t eval mode. If it is not set to eval mode then the output for the same example can be different for different runs.


I think you need to study the reason for introducing dropout/bn in NNs. Their job is to modify outputs in each iteration. Then if we disable dropouts before feedforward, why don’t we just remove them?

  
      en.wikipedia.org
  
  
    

Dilution (neural networks) 1

Dilution (also called Dropout) is a regularization technique for reducing overfitting in artificial neural networks by preventing complex co-adaptations on training data. It is an efficient way of performing model averaging with neural networks. The term dilution refers to the thinning of the weights. The term dropout refers to randomly "dropping out", or omitting, units (both hidden and visible) during the training process of a neural network. Both the thinning of weights and drop Dilution is us...