fix: keep separate hypothesis for different beam group
# What does this PR do?
It fixes the issue in diverse beam search implementation. <be>

# What was the issue?
By definition, diverse beam search is a variant of beam search which tries to group beams and introduce variance between the groups.

After successful decoding, we select beams from each group thus giving diverse solutions.

For example with beam size 3 and group size 3, we will have 1 beam in each group. And finally, if we want the top 3 suggestions, we should select the top suggestion from each group.

This condition was violated in the current implementation thus giving similar generated sequences in some cases.

Paper for reference [Diverse Beam Search](https://arxiv.org/pdf/1610.02424.pdf)


## Who can review?
@patrickvonplaten Please review changes
@ayushtiku5 could you maybe take a look here? :-)<|||||>@patrickvonplaten @ayushtiku5 doesn't seem to be available, could you please take a look?

Regarding test failure, It is related to the change itself, there are some assertions on structure of beam hypothesis array, which needs to be changed. Once change seems fine to you I can make test work.<|||||>@patrickvonplaten If you need some examples I can try to find them.
I tried to debug this issue on my custom trained weights, it would be hard to find an example on public models.<|||||>Thanks for pinging me again on this @ayubSubhaniya! I finally took some time to look a bit deeper into the PR - IMO it's not really a bug, but just a matter of how to define "beam_per_group". If `num_beams` is thought of as the total number of beams for generation then, `num_beams // num_beam_groups` is the number of beams per group. 

It is questionable whether `num_beams` in this case should represent the overall number of beams across all groups or *per* group - if I understand correctly in your opinion in should be per group. I understand this point, but the problem is that making this change now is a big backwards compatibility breaking change: Image all the users using group beam search in their pipelines now that all of a sudden get different results because the meaning of `num_beams` change. 

So, I'd prefer to not merge this PR as IMO it doesn't really fix a bug but just re-interprets the meaning of `num_beams` which is a very public API<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.