Solved by ptrblck in post #2 
              
              
                PyTorch uses the inverse scaling during training, to avoid the operation during inference, which would yield the same overall result, and is mentioned in the dropout paper in section 10: 

Another way to achieve the same effect is to scale up the retained activations by multiplying 
by 1/p at traini…