The Pegasus model generator function ignores the temperature parameter.
Loading the model using the from_pretrained method, using the 'tuner007/pegasus_summarizer' weights, and invoking the generate method with different temperatures results in always the same output text being generated regardless of the temperature.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.11.2
- Platform: Linux-5.4.0-1055-aws-x86_64-with-debian-buster-sid
- Python version: 3.7.10
- PyTorch version (GPU?): 1.9.1+cu102 (True)
- Tensorflow version (GPU?): 2.3.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help

Models:
Pegasus model (using the torch implementation)
loading the weights from 'tuner007/pegasus_summarizer'

## To reproduce

Steps to reproduce the behavior:

1. Load the model using the from_pretrained method
2. Invoke the generate method with different temperatures.

## Expected behavior
Get different output texts

<!-- A clear and concise description of what you would expect to happen. -->

Could you post a code snippet so we could take a look ? Also `temperature` parameter is for sampling so make sure to enable sampling, you could do this by passing the `do_sample=True` argument to the `generate` method, otherwise `temperature` won't have any effect.<|||||>This is the code (extracted from the url of the author of the model)

import torch
from transformers import PegasusForConditionalGeneration, PegasusTokenizer
model_name = 'tuner007/pegasus_summarizer'
torch_device = 'cuda' if torch.cuda.is_available() else 'cpu'
tokenizer = PegasusTokenizer.from_pretrained(model_name)
model = PegasusForConditionalGeneration.from_pretrained(model_name).to(torch_device)

def get_response(input_text, temp):
    batch = tokenizer([input_text],truncation=True,padding='longest',max_length=1024, return_tensors="pt").to(torch_device)
    gen_out = model.generate(**batch,max_length=128,num_beams=5, num_return_sequences=1, temperature=temp)
    output_text = tokenizer.batch_decode(gen_out, skip_special_tokens=True)
    return output_text

Adding the `do_sample=True` did modified the output taking into account the `temperature,` modifying the beam produced a different output without having to sample, Is this the intended behavior?<|||||>if sampling is `True` and `num_beams`  is > 1 then it'll do beam sampling. If you just want to do temperature sampling, you could set `num_beams=1`<|||||>What I mean is that in the absence of the `to_sample=True` parameter modifying the `num_beams` changes the output but not changing the `temperature`<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.