Hey,
I’m trying to reproduce some previous work I did with Theano in PyTorch, with RNNs.
I want to be able to mask the sequences I pass as input to an RNN based model. This should be easy enough…
However, there’s a couple of annoying issues that are bugging me:


PackedSequences inputs are only supported by RNNs, which means that I have to constantly use pack_padded_sequences and pad_packed_sequences constantly, back and forth, in order to have a model with RNN layers that interact with other types of layers.


The requirement of pack_padded_sequences of having a sorted sequence length list. This is pretty much the same as bucketing and it is conditioning training by not allowing random sapling. By having this requirement, how can I combine a DataLoader with PackedSequence without having to fully sort my dataset by length?


And finally, one last question:
Is it possible to mask the loss function natively in PyTorch?
Cheers!