I ended up not having a padding token in the vocabulary and then I padded the batch with whatever index was len(TRG.vocab)+1. Whenever I needed the pad_idx (for ignore_idx in the criterion for example) I just used len(TRG.vocab)+1. Note that I built my own Dataset and Loader so this may not work if you’re not padding your batches yourself.