You could write your own module, where you process the whole sequence, sample the mask once at the beginning and just do an element-wise multiplication after each step. Here’s a gist:
def forward(self, x):
    mask = Variable(torch.Tensor(x.size(0), self.hidden_size).fill_(self.dropout).bernoulli_())
    hidden = None
    outputs = []
    for t in x.size(1):
        output, hidden = self.rnn(x[:, t], hidden)
        outputs.append(output * mask)
    return outputs, hidden