Relaxing `PreTrainedModel` requirement in _save
# 🚀 Feature request
It's great to see that `Trainer` is becoming flexible. Each functions seems to be more self contained now making inheritance easier. I've experimented with many custom models. For instance, 
```
class Model(nn.Module):
    def __init__(self, ..):
        self.encoder = AutoModel.from_pretrained(..)
        self.custom_modules = ..
    def forward(self, **kwargs):
        output = self.encoder(**kwargs)
        # some custom operations
```
Many users are required to create custom models if they just don't want simple `SequenceClassification` head. In all cases, I have to override `_save` method because of [this line](https://github.com/huggingface/transformers/blob/d822ab636b6a14ed50f7bca0797c1de42c19de61/src/transformers/trainer.py#L1097) which explicitly puts a restriction on `Trainer` to be used with models that inherit from `PreTrainedModel`. It would be good to relax this requirement and give a warning about not using `PreTrainedModel` instead.

## Your contribution
I'll open a PR if I get approval.

I don't see anything blocking with this. Wdyt @sgugger @julien-c ?<|||||>We can give a warning but then the rest of the method will fail. Are you thinking of aborting the save entirely for models that are not `PretrainedModel`s? Also, why are you not inheriting from `PretrainedModel` in your example? Is there something limiting?

Note that Trainer is not supposed to be a generic training loop, but we can surely make it a bit more flexible.<|||||>Yes, `Trainer` is not a general loop, but it works for custom models as I've tried. Majority of its parts are generalized. `PreTrainedModel` also inherits from `nn.Module`, so users can do that, although its quite common for users to inherit from `nn.Module` directly. I'm not sure how the method will fail ? We can just add a warning instead of raising a `ValueError`. The reason why I'm saying is that users would want to do more than just what `transformers` provide out of the box (for instance justing using `AutoModel` and not `SequenceClassification` models (I'm seeing a growing interest in using such models). I think `nlp` is heading towards that direction (making everything general). This works fine for all cases, I guess:
```
from types import MethodType

def _save(self, output_dir: Optional[str] = None):
    output_dir = output_dir if output_dir is not None else self.args.output_dir
    os.makedirs(output_dir, exist_ok=True)
    logger.info("Saving model checkpoint to %s", output_dir)

    torch.save(
        {"model_state_dict": self.model.state_dict()},
        os.path.join(output_dir, "pytorch_model.bin"),
    )

    # Good practice: save your training arguments together with the trained model
    torch.save(self.args, os.path.join(output_dir, "training_args.bin"))

trainer._save = MethodType(_save, trainer)
```
Where do you think the approach may not work ? After providing the warning, its upto users if they further want to make changes by overriding this method (they would know that `transformers` is not responsible anymore since its not a `PreTrainedModel`. Current method completely breaks the training due to `ValueError`.
This is optional, I felt that it would be useful to have. I'll open a PR if you approve.<|||||>`save_pretrained` does more than the method you mention, but we could refactor the code inside to work with all models probably. I don't see any place it uses specific stuff from `PretrainedModel`. The thing we don't want is to add and maintain too generic code, but if it's easy enough I see no objection.

You didn't tell me why subclassing `PreTrainedModel` did not work however ;-) That is what I would expect a user building a custom model using transformers to do .<|||||>The `PreTrainedModel` is a generic class amongst all models in `transformers`, all classes pertaining to it comply in terms of the methods it provides and can use functionalities such as `init_weights`, `prune_heads`. They might not work for custom models. For instance, some methods require `.config.` attribute which custom models may not directly have. I guess one can define their custom model to be exactly what `PreTrainedModel` requires them to be (haven't looked into that), but that would be asking users to read through what `PreTrainedModel` expects or maybe specifying in docs. It's totally up to you what you expect the users to do in case they use custom models.<|||||>After some internal discussion with @julien-c we will lower the requirement from `PreTrainedModel` to some lower abstractclass/protocol so the user knows exactly what they have to implement for their model to work seamlessly with `Trainer`. I will work on this end of this week beginning of next. <|||||>Sounds good. I'll look forward to that part then.