I thought about this a bit now, and I have one more question if you can answer: the attention mask takes care of what tokens the model attends to. If we do not pass the attention_mask to the model, does the -100 value takes cares of that as well?
As you said, it indicates to the loss function what tokens to ignore. But the decision to attend to specific tokens or not is something different, isn’t it?