[CodeGen] RuntimeError: where expected condition to be a boolean tensor, but got a tensor with dtype Half
### System Info

transformers==4.21.2
torch==1.10.2+cu113

HW: 12 cpu, 90gb ram, 1xA100-80g

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

**The following works with CodeGen-350M-mono / CodeGen-6B-mono**

```
import torch
from transformers import AutoTokenizer, CodeGenForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Salesforce/codegen-350M-mono")
model = CodeGenForCausalLM.from_pretrained("Salesforce/codegen-350M-mono", device_map="auto")
```

**But throws the RuntimeError for CodeGen-16B-mono**

```
import torch
from transformers import AutoTokenizer, CodeGenForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Salesforce/codegen-16B-mono")
model = CodeGenForCausalLM.from_pretrained("Salesforce/codegen-16B-mono", device_map="auto")
```

### Expected behavior

Expect the 16B model to work as well.
cc @younesbelkada <|||||>Another way to reproduce is to use https://github.com/huggingface/transformers-bloom-inference

and run:

```
make codegen-mono
```<|||||>Hi @jrdzha 
Thanks so much for your issue! Unfortunately I did not managed to reproduce your issue. How are you getting this error? Could you provide me a full example script please?
Here is the snippet I used: 
```
import torch
from transformers import AutoTokenizer, CodeGenForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Salesforce/codegen-16B-mono")
model = CodeGenForCausalLM.from_pretrained("Salesforce/codegen-16B-mono", device_map="auto")

text = "def main():"
inputs = tokenizer(text, return_tensors="pt")

model.generate(**inputs)
```
Also can you make sure to use the latest version of `accelerate` ? `pip install --upgrade accelerate` + is there anything blocking on your side not to use the latest version of `transformers`? `pip install --upgrade transformers` <|||||>Seems also to be a duplicate of https://github.com/arrmansa/Basic-UI-for-GPT-J-6B-with-low-vram/issues/4 I also ran the experiment with `torch==1.10` and not getting any error ! <|||||>It seems this issue is inconsistent. I did have it work once, but without changing any code... I'll spend some more time trying to reproduce this more reliably...<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.