Solved by ptrblck in post #2 
              
              
                I think you have a typo in your first model, as you are not using the Dropout layer in forward: 
x - self.dropout(x)

This should probably be an assignment instead of a subtraction. 
You will get exactly the same output, if you fix this typo and seed all operations accordingly: 
torch.manual_seed(28…