I’m implementing as follows, where the h1_kernel, c1_kernel, etc are hidden and cell state kernels, for learning initial hidden layers.  (I prefer to learn kernels so that I can easily change batch sizes later; the full hidden and cell states are just repetitions of the learned kernels.)
    def forward(self, inputs, lengths, batch_size):
        self.h1, self.c1   = self.init_hidden(batch_size, self.h1_kernel, self.c1_kernel)
        self.h2, self.c2   = self.init_hidden(batch_size, self.h2_kernel, self.c2_kernel)

        pack1              = nn.utils.rnn.pack_padded_sequence(inputs, lengths, batch_first=True)
        out1, _            = self.lstm1(pack1, (self.h1, self.c1))
        pad1               = nn.utils.rnn.pad_packed_sequence(out1, batch_first=True)[0]
        drop1              = self.drop(pad1.data) 

        pack2              = nn.utils.rnn.pack_padded_sequence(drop1, lengths, batch_first=True)
        out2, _            = self.lstm2(pack2, (self.h2, self.c2))
        pad2               = nn.utils.rnn.pad_packed_sequence(out2, batch_first=True)
    
        dense_out          = self.dense(pad2[0])
        pack_dense         = nn.utils.rnn.pack_padded_sequence(dense_out, lengths, batch_first=True)
        pad_dense          = nn.utils.rnn.pad_packed_sequence(pack_dense, batch_first=True)

In this case, the dropout is set elsewhere to be 0, i.e., present, but de-activated.
If I remove the dropout altogether (and adjust the rest of the forward accordingly) the observed behavior in training is significantly (and repeatably) different.  Absent dropout works better than deactivated dropout.
Can anyone please shed some light on this?  Why does this happen?  What is the correct way to do this if I want actual non-zero dropout?