Why use dropout in feedforward? Maybe I am just confused by what is the point of using dropout in the feed-forward? Wouldn't be better to forward the input with the whole network and then use the dropout only in the back-prop to limit the modifications of weights and biases of a limited number of neurons and hence reduce overfitting?
Thank you in advance

The purpose of dropout is to average over the exponentially many thinned networks that arise from randomly removing units. Your suggestion of dropping out units during backprop only doesn’t achieve this, because all units were used during the forward pass.
