My particular problem with 1. is that I can’t really see a good spot to send the tensors to GPU before doing pad_sequence. I want to return a Tensor from collate_fn (if I return a list I again get strange behaviour with num_workers > 0), so it seems to me (at least for now) I need to pad in collate_fn. On the other hand, I can’t/shouldn’t send Tensors to GPU in collate_fn to avoid problems.
You mind sharing a piece of code of how are you doing it? Could be easier than explaining it in text.