Machine Learning, Deep Learning, Artificial Intelligence, Regularization, Deep Learning Book.
be zero. Re-normalization to make sure all the probabilities sum to 1. The advantage for dropout is that first term can be approximated in one pass of the complete model by dividing the weight values by the keep probability (weight scaling inference rule). The motivation behind this is to capture