From what I gather, ignore_index works for the entire minibatch and not per sample.
I’m trying to mask things because I don’t want my network to learn that it can just say ‘pad’ everywhere and get a lower loss for shorter sentences.