Hi,
looking at the TransformerEncoderLayer 2 and TransformerDecoderLayer code, it seems that at inference time dropout is applied without change.
I thought that the dropout is not used at inference time. Is it correct or am I missing something?