llama model can't generate EOS
### System Info

python                    3.8.16
torch                     1.13.1
transformers              4.28.0

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
from transformers import LlamaTokenizer, LlamaForCausalLM, GenerationConfig

# both model have same behavior
model_path = "luodian/llama-7b-hf"
# model_path = "huggyllama/llama-7b"

model = LlamaForCausalLM.from_pretrained(model_path)
tokenizer = LlamaTokenizer.from_pretrained(model_path)
tokenizer.pad_token = tokenizer.eos_token

text = ["Translate english to chinese: I love you.", "What is your name:"]
a = tokenizer(text, return_tensors='pt',padding="longest")
print(model.generate(**a, max_new_tokens=64))
```

### Expected behavior

The llama model's generate method doesn't generate any EOS token under any circumstances
cc @ArthurZucker <|||||>Hey!
This seems to be a bit similar to #23175. 
When the `generate` function is called, it should stop once the `eos_token` (which is `2`). 
If the model does not predict it, then the generate function will not stop. This can come from the training, but is most probably not an issue with the `generate` function. 

You can check the original behaviour here: https://github.com/facebookresearch/llama/blob/main/llama/generation.py you'll see that it does not stop on the `eos` token. <|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.<|||||>what was the soln?