 When using model.generate, it does not stop at eos_token, but instead continues until the maximum length.
### System Info

ubuntu20.04
transformers==4.29.0

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

use my own model, infer by


```shell
        output_texts = model.generate(
            input_ids=input_ids,
            attention_mask=attention_mask,
            pad_token_id= tokenizer.eos_token_id,
           eos_token_id= tokenizer.eos_token_id,
            max_new_tokens=500, do_sample=False,
            top_k=30, top_p=0.85, temperature=0.3, repetition_penalty=1.2)
```
output like:
```shell
因此需要进行额外的优化。</s></s></s></s></s></s></s></s></s></s></s></s></s></s></s></s></s></s></s></s></s></s></s></s></s></s>
```



### Expected behavior

should end by eos token but not


Hey @bestpredicts 👋 

I agree that `</s>` should not be generated. However, I need a stand-alone short reproducer to understand what's truly going on. We have many Llama checkpoints that are not compatible with `transformers`.<|||||>@bestpredicts I have same question, do you solved it?<|||||>> Hey @bestpredicts 👋
> 
> I agree that `</s>` should not be generated. However, I need a stand-alone short reproducer to understand what's truly going on. We have many Llama checkpoints that are not compatible with `transformers`.

me too,same question.<|||||>You problem can come from the `model.eos_token_id` that is not the correct one (wild guess) but we need a minimal reproducer to help you. <|||||>Edit: have deleted this comment, because think the issue I was seeing was just the EOS was outputted as very unlikely by the models for some reason. (Check the history for the code snippet.)<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.