Differences between S2T and Wav2Vec2
# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

There seem to be differences between S2T and Wav2Vec2 which are hard to understand reason about that may be fixable.

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

Adding something like AutomaticSpeechRecognitionPipeline might be desirable and would be hard to do in current state. If/When new multimodal models are added it is going to add more and more complexity. Aiming for consistent API is desirable IMO.

## Description

- There are no `AutoProcessor.from_pretrained`.
- Wav2Vec2 cannot use `skip_special_tokens` for the decode variant (because it skips `<pad>` tokens early leading to removing all duplicate letters from the output. IMO it's a "bug" as `<pad>` in the context of CTC is not a special_tokens (at least until letters are resolved). 
- Wav2Vec2 uses 1 forward pass, where S2T uses  generate function. It would be nice, if there could be 1 interface only. (maybe just overload the `Wav2Vec2ForCTC.generate` ?
- S2T overloads `input_ids` with float tensors when `generating`, which works in practice but does seem like a piggy-back of the generate function and is definitely confusing to use. If `generate` is generic enough maybe then `input_ids` should be renamed to reflect that (`input_ids` are IDs everywhere in the rest of transformers. It could be a simple internal variable rename, I don't imply we should change any function signature anywhere, just that the variable is not necessarily IDs. Isn't it a bit like `inputs_embed` ?
- Wav2Vec2Processor returns 'input_values' where S2TProcessor returns 'input_features'. They seem (at least in appearance) to be the same. Would it be better to use only 1 name if they do ?

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->

Happy to contribute with PRs but I lack the more general view to be sure about what direction to take, and where are the "better" fixes.
@patrickvonplaten 
Who should I tag for S2T ?<|||||>@patil-suraj for s2t<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.