Weird outputs by `opus-mt-en-es`
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: TPU VM
- Python version: 3.8.10

### Who can help
@patrickvonplaten @patil-suraj 

## Information
Model I am using: FlaxMarianMTModel


## To reproduce
I used this code for beam size 2 and 4. Funnily the outputs looked almost same just that Oh changed to no in beam_size=2

```
from transformers import MarianTokenizer, FlaxMarianMTModel
model = FlaxMarianMTModel.from_pretrained('Helsinki-NLP/opus-mt-en-es', from_pt=True)
tokenizer = MarianTokenizer.from_pretrained('Helsinki-NLP/opus-mt-en-es')
text = "Living Room, The Sheridan House! Your Minneapolis Home!"
input_ids = tokenizer(text, max_length=64, return_tensors='jax', truncation=True)
sequences = model.generate(**input_ids, early_stopping=True, max_length=64, num_beams=2).sequences
tokenizer.batch_decode(sequences, skip_special_tokens=True, max_length=64)
```
For num_beams = 2 output:
'Sala, ¡La Casa Sheridan, tu hogar de Minneapolis, ¡No, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no'

For num_beams = 4 output:
'¡Sala de estar, la Casa Sheridan, tu hogar de Minneapolis, ¡Oh, ¡Oh, oh, oh, oh, oh, oh, oh, oh, oh, oh, oh, oh, oh, oh, oh, oh, oh, oh, oh, oh,'

## Expected behavior
Shouldn't give 'oh' or 'no' in outputs.
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>Fixed via #12662 