GPT2 and -100 in input_ids
### System Info

I post all detail maybe it is expected behavior. 

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

Hi Folks,

I remember it used to work, but I may be missing something.  I noticed if I set the value in input_ids to -100.  
i.e., standard ignore value.   GPT-2 crash is essentially used as an index value where it should ignore it. 
Is there anything changed in GPT-2 model recently in the code base?  I'm not 100% sure is a bug or not. 

This led to a crash.   

```
attention_mask = batch['attention_mask']
mask_inverted = ~attention_mask.bool()
input_ids = batch["input_ids"]
input_ids = input_ids.masked_fill(mask_inverted == 1, -100).contiguous()
```

Thank you.

### Expected behavior

Expected behavior 
value in the input_ids should [id_n, id_n+1 etc,  eos_id,  -100]
You can put -100 in the `labels` so they are ignored by the loss, but in the inputs as it's not a valid index for the embedding matrix.

Also this kind of questions is better suite for the [forums](https://discuss.huggingface.co/) as we keep GitHub issues for bugs and feature requests only.<|||||>Thank you very much for confirming, strange ... I have an old code that did work before I think it was masking -100. anyway looks like im a bit off here. : ) thank you! 