Beam search not always better Greedy search
@patrickvonplaten

I was recently tinkering around the generation strategies for a decoder, and came across this [blog post](https://huggingface.co/blog/how-to-generate) on decoding methods. It was quite useful to clarify how generation works for transformers. Thank you for this excellent work.
I notice that in the section on Beam Search, it is stated that:

> **Beam search will always find an output sequence with higher probability than greedy search**, but is not guaranteed to find the most likely output.

It seems to me that the highlighted part is not true, which can illustrated by the following, extended from the example in your blog post.
 
![image](https://user-images.githubusercontent.com/3142085/228549175-d37bb8af-ed62-44bf-93f8-38d22bf6aba3.png)

Basically if better beams at a certain time are followed by more tokens in which the preference of tokens is not that strong (distribution being flatter), then the greedy result could have a higher probability, and being missed out by the beam search.
I got curious about this claim, because I tried beam search and greedy search in a vision encoder decoder model (Donut), and got lower probability generations quite often with beam search.

What do you think? Is there something else that I over-looked in this example?

Thank you and have a nice day.
Actually, copied to the issues section of the blog.