Wav2Vec2 models must either throw or deal with add_apater
Using `Wav2Vec2Model` models with `config.add_adapter=True` as `Wav2Vec2ForCTC` (for instance) doesn't throw on creation, but eventually fails due to mismatch in the output dimension (`config.output_hidden_size`) and the `lm_head` layer put on top of it (`config.output_hidden`).

# What does this PR do?

I propose:

  1. to fix this for two sub-models that only rely on the last output (by using `output_hidden_size` when `add_adapter` is set), and 
  2. to raise an exception for the other types of models (that try to average multiple layers). Raising an explicit error is better than failing during the forward pass.

Another option is:

  3. to always throw if add_adapter is set.

## Before submitting
/!\ This is a draft pull request for review of the general proposal. I did not test this code or write tests yet. I edited the file online directly on github.


## Who can review?

Cc @patrickvonplaten since this is related to the robust-speech-event (community event)
_The documentation is not available anymore as the PR was closed or merged._<|||||>Based on the current CI results:
1. This needs reformatting if we want to merge.
2. All the Wav2Vec2-like models should mirror this change.
3. Ideally, a test would be written to ensure this doesn't regress.

But I still want to have feedback on the preferred approach before considering to go through all the hoops.<|||||>Hey @FremyCompany,

Thanks for the PR. I think it's a welcoming change. Could you run:

```make fix-copies``` so that this change is automatically copied to other varients of Wav2Vec2?<|||||>There is an issue with the changes propagated by the fix-copies command.
```
AttributeError: 'UniSpeechSatConfig' object has no attribute 'add_adapter'
```

I'm not sure how that should be fixed. Any idea?<|||||>I went for an `hasattr` check. 
(Assumption: if the key is absent, it defaults to False).