I’m assuming you’re calling the encoder from  within  the forward function.

Yes, this code is all in the forward function for the instruction encoder Module (the Module object is an attribute of another Module who is wrapped in DataParallel, and its forward call is called during the top-level forward call. It is very modular code!). The forward call for this model takes as input a list of string vectors seqs (List[List[str]]), and just before the call I posted, I convert them in to lists of ints:
batch_indices: List[List[int]] = [[self._embedder.get_index(tok) for tok in seq] for seq in seqs]
seq_lens: List[int] = [len(instruction) for instruction in instructions]

I think I know what the issue is – does DataParallel require that the input to the forward calls be tensors so it can distribute them?