Hi @valhalla , thanks for the explanation. I am also interested in this topic. Would you take a look at my question? Thanks in advance.
what @sanaz wanted to point out is that:
the ground-truth label based on your code:
[-100, -100, …, 103, …, -100 ]
the ground-truth label based on sanaz’s modified code:
[-100, -100, …, token_id(7th token), …, -100 ]
I know that tokens represented by -100 will be ignored, but for the token (in this case, 7th token) to be predicted, we still use ‘103’ as the ground-truth label?