I think you should sample the bernoulli using:
Variable(torch.bernoulli(torch.Tensor(x.size(0), self.hidden_size).fill_(self.dropout)))

otherwise your mask won’t be correct when using self.dropout = 1.0 or self.dropout = 0.0. I just ran into it, and found that it is an issue on GitHub 64.