Missing parameters when iterating over `module.parameters()`
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: `4.15.0`
- Python
  - Implementation: CPython
  - Version       : 3.8.12
- Platform
  - OS          : Darwin
  - Release     : 20.6.0
  - Machine     : arm64
  - Architecture: 64bit
- PyTorch version (GPU?): `torch==1.10.1` no GPU
- Tensorflow version (GPU?): None
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@patrickvonplaten, @LysandreJik

## Information
For some reason, the parameter count of an entire `GPT2LMHeadModel` is not considering the `lm_head`. This behavior might not be only specific to this class, however, this is the one I am working with.

## To reproduce
```python
from transformers import AutoModelForCausalLM


def count_params(module):
    return sum(p.numel() for p in module.parameters())


name = "gpt2"

model = AutoModelForCausalLM.from_pretrained(name)

print(f"Whole model: {count_params(model)}")
print(f"Transformer: {count_params(model.transformer)}")
print(f"Head: {count_params(model.lm_head)}")
```
The standard output:
```bash
Whole model: 124439808
Transformer: 124439808
Head: 38597376
```

The core of the problem is that the `lm_head.weight` is not inside of `model.named_parameters()`.

## Expected behavior
The expected behavior is for the total number of parameters to be equal to the parameters inside of the `transformer` backbone PLUS the number of parameters inside of `lm_head`.

See relevant code here: 
https://github.com/huggingface/transformers/blob/1c121916f3adee769eb43d4656b621be60427bbd/src/transformers/models/gpt2/modeling_gpt2.py#L951-L952


Hey @jankrepl,

`self.lm_head` is identical to `self.transformer.wte` since those weights are tied (input and output word embeddings are the same for a lot of Transformer models) which is why `self.lm_head` doesn't show up here I think <|||||>```python
from transformers import AutoModelForCausalLM


def count_params(module):
    return sum(p.numel() for p in module.parameters())


name = "gpt2"

model = AutoModelForCausalLM.from_pretrained(name, tie_word_embeddings=False)

print(f"Whole model: {count_params(model)}")
print(f"Transformer: {count_params(model.transformer)}")
print(f"Head: {count_params(model.lm_head)}")
```

should give

```
Whole model: 163037184
Transformer: 124439808
Head: 38597376
```<|||||>Oh wow! What a nice trick to reduce the number of parameters!

Thank you for the quick help:)