Solved by ptrblck in post #2 
              
              
                Yes, it should be disabled: 
# no dropout
model = nn.LSTM(10, 10, 2, dropout=0.0)
x = torch.randn(10, 10, 10)
out1, _ = model(x)
out2, _ = model(x)
print((out1 - out2).abs().max())
> tensor(0., grad_fn=<MaxBackward1>)

# dropout
model = nn.LSTM(10, 10, 2, dropout=0.5)
x = torch.randn(10, 10, 10)
out…