HI guys,
I have a question about the behaviour of dropout layer during training and evaluation. I remember reading in a paper that because dropout leave out some units during training. During evaluation, the out going weights of dropout layer need to be reduced an amount corresponding to the dropout rate. For instance, if the dropout rate is 0.5, then the out-going weights need to be reduced by 2, because during evaluation, we effectively have twice the number of units.
So my question is, is this kind of weight scaling mechanism included in the dropout layer in pytorch as well?
Cheers and thanks a lot for your help.
Shuokai