t5 model's decoder do not use EncDecAttention.key & value in text generation task
### System Info

- `transformers` version: 4.20.1
- Platform: macOS-12.4-arm64-arm-64bit
- Python version: 3.9.10
- Huggingface_hub version: 0.8.1
- PyTorch version (GPU?): 1.13.0.dev20220709 (False)
- Tensorflow version (GPU?): 2.8.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@patrickvonplaten, @Narsil, @gante

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I use `transformers.T5ForConditionalGeneration` to do inference in text generation task(NaturalQuestions). We all know that the longer the answer is the more times the `decoder` goes `forward`. But when I tried to log the forward in each layer. I found that the model goes `encoder` once, `A type decoder` once,  `B type decoder` many times based on the answer length.

### Expected behavior

`A type decoder` use EncDecAttention. q,k,v,o

![image](https://user-images.githubusercontent.com/84232793/197341985-bf12427d-2323-475c-865a-e3a0a61d7bde.png)


But `B type decoder` only use EncDecAttention. q,o
![image](https://user-images.githubusercontent.com/84232793/197341988-d9466cfa-c217-4cc2-ae73-296b50d4c77c.png)

Many thanks!! :)
Hi @CaffreyR 👋 Our models, when used for generation purposes, have a cache that stores repeated computations (key and value of the attentions) turned on by default. If you'd wish to see the full (redundant) computations being executed, try using the model with `use_cache=False`.<|||||>Great! Thanks for your help!