AttributeError: 'T5ForConditionalGeneration' object has no attribute 'linear'
## Environment info

- `transformers` version: 4.9.1
- Platform: Linux-5.4.0-84-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.8.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?:NO
- Using distributed or parallel set-up in script?:NO

Models:

- model_name = "flexudy/t5-small-wav2vec2-grammar-fixer"

Library:
- text generation: @patrickvonplaten

Documentation: @sgugger

## Information

I am trying to do the static quantization on the T5 model(flexudy/t5-small-wav2vec2-grammar-fixer) for reducing the inference time.

## code used
```
import torch
import transformers
from transformers import T5Tokenizer, T5ForConditionalGeneration
model_name             =  "flexudy/t5-small-wav2vec2-grammar-fixer"
torch_device              =  'cuda' if torch.cuda.is_available() else 'cpu'
model                        = T5ForConditionalGeneration.from_pretrained(model_name).to(torch_device)
model.eval()
model.qconfig            =  torch.quantization.get_default_qconfig('fbgemm')
model_fused             = torch.quantization.fuse_modules(model,[['linear', 'linear']])
```


## Output
AttributeError: 'T5ForConditionalGeneration' object has no attribute 'linear'

## Expected behavior

Model modules should be fused for the quantization

Hey @pradeepdev-1995,

I'm not super familiar with `torch.quantization.fuse_models(...)`. Could you explain a bit more what those lines do:

```
model.qconfig            =  torch.quantization.get_default_qconfig('fbgemm')
model_fused             = torch.quantization.fuse_modules(model,[['linear', 'linear']])
```

?<|||||>@patrickvonplaten 
could you please check the official documentation https://pytorch.org/docs/stable/quantization.html<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.