Add return type hint to AutoModel.from_pretained
### Feature request

I think the ergonomics of using, e.g. `AutoModelForSequenceClassification.from_pretrained(...)` can be improved.

Consider the following example:
```python
model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased")
```
It's quite hard to reason about `model` since we don't really know _what_ it is without inspecting it at runtime...

More concretely, I wanted to freeze BERT's internal parameters but not those of the classifier layer.



### Motivation

The productivity of developers using the automodel API can be improved; code can be checked by linters more thoroughly etc.

### Your contribution

I'd like to open a PR; but am not sure what the best return type is for `AutoModelForSequenceClassification.from_pretrained`.

If we could discuss this here and reach some sort of consensus that this is desirable I will draft a pull request.
Hi @JosephSBoyle, thanks for raising this issue. 

Could you give some more information about the behaviour you expect and expand on "we don't really know what it is without inspecting it at runtime..." ? As I understand the issue, there's three different points being raised:
* Knowing what the model "is"
* How to easily modify the model's behaviour
* IDE integration i.e. linters

Is this correct? <|||||>Hia @amyeroberts, apologies my writing in the original issue is a bit unclear.

To elaborate on my "not knowing what it is" statement, basically my point is that it's quite difficult to know the type of whatever model instance is returned when you call `from_pretrained`. The "at runtime" part of that statement was in reference to basically inspecting the returned instance e.g. in a breakpoint; which is what I ended up doing.

I think that my points can best be summarized as: "without the return type the `AutoModelForSequenceClassification.from_pretrained` method is difficult to use effectively."
<|||||>The `AutoXxx.from_pretained(checkpoint)` API is essentially a factory method, loading the architecture / model specified by `checkpoint`. So, for AutoModelForSequenceClassification, any model which has a [sequence classification head](https://github.com/huggingface/transformers/blob/d6f1da6b7169e3b2bcc2fcdc91a19171ecafeb88/src/transformers/models/auto/modeling_auto.py#L641) can be returned. As such, there isn't a predefined type (other than being a subclass of `PreTrainedModel`). 

After loading a model, it's possible to check its class:
```python
In [1]: from transformers import AutoModelForSequenceClassification

In [2]: model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased")

In [3]: type(model)
Out[3]: transformers.models.bert.modeling_bert.BertForSequenceClassification
```

Specific model architectures can be loaded directly too: 
```python
In [1]: from transformers import BertForSequenceClassification

In [2]: model = BertForSequenceClassification.from_pretrained("bert-base-uncased")

In [3]: type(model)
Out[3]: transformers.models.bert.modeling_bert.BertForSequenceClassification
```

From the model config, it's possible to find which model architecture will be loaded e.g. [here](https://huggingface.co/bert-base-uncased/blob/0a6aa9128b6194f4f3c4db429b6cb4891cdb421b/config.json#L3) for `bert-base-uncased`. Note: for this checkpoint, all of the weights for the base model would be loaded in, the weights for the language modeling head discarded, and weights for the classification head randomly initialised. <|||||>Mhmm, I understand that the return type varies based on the first arg. as you describe. Since we know that it's a subclass of `PreTrainedModel` I think we should at least add that in as the return type, something like this perhaps:

```python
# auto_factory.py
from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from ...modeling_utils import PreTrainedModel

...

def from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs) -> type[PreTrainedModel]:
    ...
```

**Edit:** corrected the rtype to `type[PreTrainedModel]`, as the returned type will be a subclass of this type.

---------------------------------------------------


Your first cell is actually what I ended up doing to find the type of the returned instance, this is actually what I meant by my earlier runtime comment.<|||||>Except that this type hint does not help anyone understand the result, so is it really useful to bloat the code to add it?<|||||>Is it not better to know that the returned instance is a `PreTrainedModel` rather than literally `Any`? @sgugger <|||||>Given the fact that the class is `AutoModel`, I don't think anyone will think in good faith this will return anything else than a model.<|||||>I think there's some misunderstanding here, nobody thinks that this model is returning anything other than a model.

The purpose of adding a type hint is to enable things like static analysis tools to work properly, which they can't do without knowledge of the return type.



<|||||>I think there is some misunderstanding indeed. Transformers does not support any static analysis tool like Mypy and never will, as it would require us to add type annotations that bloat the code. In all our experiments this makes the code harder to read without ever catching any useful bug.

We only use type annotations when useful for the doc (in particular seeing the signature in an IDE with type annotations when an argument's type is not obvious) but that is all.<|||||>I didn't ask for MyPy support, just a single type hint. Static analysis includes things like linting which are what I'm talking about.

For a single type hint you get:

- Attribute and method completion for attrs of `PreTrainedModel`
- Linting, which can for instance, identify when any of the methods are called with the incorrect signature.
- Better readability: programmers don't have to dig through the library to figure out that no matter what the first argument to `from_pretrained` is, they will recieve a subclass of the same class.

Moreover, you reduce the cognitive burden on users who don't have the entire API of `PreTrainedModel` memorized to heart.<|||||>Except that it's not just a single type hint. The `from_pretrained` method in `auto_factory` can either return a `PreTrainedModel`, a `TFPreTrainedModel` or a `FlaxPreTrainedModel` depending on the class it was used with. If you find a way to have a simple type hint, we will of course merge such a PR, but I don't think it's easy to add.<|||||>Hi @sgugger, thank you for the explanation - I wasn't aware that there were multiple possible return types. 