Avoid nan during sampling in generate()
# What does this PR do?

Fix CI test error
```bash
            # sample
            probs = nn.functional.softmax(next_token_scores, dim=-1)
>           next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1)
E           RuntimeError: probability tensor contains either `inf`, `nan` or element < 0
```
in
https://github.com/huggingface/transformers/runs/6959698965?check_suite_focus=true

The test `test_sample_generate` may still fail at
https://github.com/huggingface/transformers/blob/8f400775fc5bc1011a2674dcfd5408d30d69f678/tests/generation/test_generation_utils.py#L711
for some unknown reason. I think it is better to investigate this in another PR.
I have some doubts here, as this will make all tokens having equal probability to be sampled. But with all `-inf`, nothing could be sampled which leads to error. I feel there is no well-defined expected results in such edge cases.<|||||>_The documentation is not available anymore as the PR was closed or merged._<|||||>Yes, that happens only when all `-inf` along the vocab dim. I will close this PR, and we have to maybe create a doc with all possible flaky tests :-)