How can I fine-tune other languages in trocr? CER over 1
### System Info

I want to do fine-tuning using trocr.
There is a problem that CER exceeds 1.
The correct answer for the dataset is one word, but it seems that the predict value is too long.

What's the problem? Should I implement the new language with the following code?

I've been learning several times, but I've found that I'm making long sentences about text.

So there is a problem that CER goes over 1.

The link below is a code that only weights other languages without fine-tuning.

The predicted characters are quite long compared to the existing characters, and I wonder if there is a problem with EOS.

https://colab.research.google.com/drive/1ovc-9aXsYKDXAfrO0FUtoiLVkXEdrz6s?usp=sharing

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

https://colab.research.google.com/drive/1ovc-9aXsYKDXAfrO0FUtoiLVkXEdrz6s?usp=sharing

### Expected behavior

CER over 1
Hi,

Refer to this thread: https://github.com/huggingface/transformers/issues/18163. Also, please ask such questions on our [forum](https://discuss.huggingface.co/), as we'd like to keep Github issues for bugs/feature requests.

Thanks!