[Generation] `length_penalty` means `beam_alpha`
https://github.com/huggingface/transformers/blob/508baf194313c397345af868202404e285494a28/src/transformers/generation_utils.py#L949-L952
https://github.com/huggingface/transformers/blob/508baf194313c397345af868202404e285494a28/src/transformers/generation_beam_search.py#L829

There are two issues about `length_penalty`:
1. Actually, `length_penalty=1` does **NOT** mean no penalty. It is no penalty when `length_penalty=0`, right?
2. This is different from the [`beam_alpha` paper](https://arxiv.org/pdf/1609.08144.pdf), why?
    <img width="632" alt="image" src="https://user-images.githubusercontent.com/42370681/165103163-b5946237-b03e-4a24-bb3e-2122a9dacec2.png">

See https://github.com/huggingface/transformers/issues/4915  https://github.com/huggingface/transformers/issues/4918#issuecomment-681985118  https://github.com/huggingface/transformers/issues/14768<|||||>cc @patrickvonplaten 
https://github.com/huggingface/transformers/blame/508baf194313c397345af868202404e285494a28/src/transformers/generation_beam_search.py#L829<|||||>Hey @ShaneTian,

Good catch! I think that's exactly right - we should change the docs here to:

```py
             length_penalty (`float`, *optional*, defaults to 1.0): 
                 Exponential penalty to the length. 1.0 means that the beam score is penalized by the sequence length. 0.0 means no penalty. Set to values < 0.0 in order to encourage the 
                 model to generate longer sequences, to a value > 0.0 in order to encourage the model to produce shorter 
                 sequences. 
```

Would you like to open a pull request for this?<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.<|||||>@ShaneTian note that the length penalty is different from the "alpha" penalty simply because we discovered it first ;-) Would be too difficult to change now though<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.