AttributeError: type object has no attribute 'forward'
### System Info

- `transformers` version: 4.21.1
- Platform: Linux-5.4.0
- Python version: 3.8.8
- Huggingface_hub version: 0.10.1
- PyTorch version (GPU?): 1.12.1+cu113 (True)
- Tensorflow version (GPU?): 2.8.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help?

@ArthurZucker and @gante 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

```py
from transformers import BertConfig, TFBertForSequenceClassification, AutoTokenizer, BertConfig
import tensorflow as tf

class CustomTFModel(TFBertForSequenceClassification):
    def __init__(self, config: BertConfig, *inputs, **kwargs):
        super().__init__(config, *inputs, **kwargs)

tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
conf = BertConfig.from_pretrained('bert-base-uncased')
model = CustomTFModel(conf)
#model = TFBertForSequenceClassification.from_pretrained('bert-base-uncased')
labels = tf.convert_to_tensor([0,1,2])

ds = tf.data.Dataset.from_tensor_slices({'input_ids':model.dummy_inputs['input_ids'],'labels': labels}).batch(1)

model.compile(
    tf.keras.optimizers.Adam(learning_rate=3e-5),
    metrics='accuracy'
    )

model.fit(
    ds,
    epochs=1,
    shuffle=True,
    verbose=1,
)
```

### Expected behavior

when I execute this code I get the error:
```bash
File "issue.py", line 21, in <module>
    model.fit(
  File "/home/ubuntu/anaconda3/envs/tf2_mattia/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/home/ubuntu/anaconda3/envs/tf2_mattia/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py", line 1147, in autograph_handler
    raise e.ag_error_metadata.to_exception(e)
AttributeError: in user code:

    File "/home/ubuntu/anaconda3/envs/tf2_mattia/lib/python3.8/site-packages/keras/engine/training.py", line 1021, in train_function  *
        return step_function(self, iterator)
    File "/home/ubuntu/anaconda3/envs/tf2_mattia/lib/python3.8/site-packages/keras/engine/training.py", line 1010, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/home/ubuntu/anaconda3/envs/tf2_mattia/lib/python3.8/site-packages/keras/engine/training.py", line 1000, in run_step  **
        outputs = model.train_step(data)
    File "/home/ubuntu/anaconda3/envs/tf2_mattia/lib/python3.8/site-packages/transformers/modeling_tf_utils.py", line 1353, in train_step
        label_kwargs = find_labels(self.__class__)
    File "/home/ubuntu/anaconda3/envs/tf2_mattia/lib/python3.8/site-packages/transformers/utils/generic.py", line 309, in find_labels
        signature = inspect.signature(model_class.forward)

    AttributeError: type object 'CustomTFModel' has no attribute 'forward'
```
﻿﻿﻿I would expect that the `type object 'CustomTFModel' has no attribute 'forward'` error would not be generated as the `CustomTFModel` class should be identical to `TFBertForSequenceClassification` or am I wrong? 

if you use `TFBertForSequenceClassification` instead (the commented line) the error is not triggered.
Hey! Thanks for posting. 
You meed to re-define the `forward` method. The following script works as expected
```python 
from transformers import BertConfig, TFBertForSequenceClassification, AutoTokenizer, BertConfig
import tensorflow as tf

class CustomTFModel(TFBertForSequenceClassification):
    def __init__(self, config: BertConfig, *inputs, **kwargs):
        super().__init__(config, *inputs, **kwargs)
    def forward(self, **kwargs):
        super().call(**kwargs)

tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
conf = BertConfig.from_pretrained('bert-base-uncased')
model = CustomTFModel(conf)
#model = TFBertForSequenceClassification.from_pretrained('bert-base-uncased')
labels = tf.convert_to_tensor([0,1,2])

ds = tf.data.Dataset.from_tensor_slices({'input_ids':model.dummy_inputs['input_ids'],'labels': labels}).batch(1)

model.compile(
    tf.keras.optimizers.Adam(learning_rate=3e-5),
    metrics='accuracy'
    )
model.fit(
    ds,
    epochs=1,
    shuffle=True,
    verbose=1,
)
```
(that is a quick fix, for TF it should be checking for a `call` method).
We check whether the model name starts with `TF`. So `TFCustomModel` will also work without redefining the function.
```python 
class TFCustomModel(TFBertForSequenceClassification):
    def __init__(self, config: BertConfig, *inputs, **kwargs):
        super().__init__(config, *inputs, **kwargs)
```
This probably was not clear so we might need to update the doc @gante <|||||>Hey! Thank you very much for your quick reply. Now it is working correctly.

I was using the quick fix you proposed too, but I noticed that during training the metrics that are passed inside `model.compile` are not evaluated. I don't know if this is a problem on your side or if it is due to tensorflow, I tried to find out what is the problem but I couldn't understand it.<|||||>Hey all!

Technically, this problem __can__ be solved without documentation (which is tricky to sort for this particular problem). If we replace the class name check with an inheritance check (e.g. [here](https://github.com/huggingface/transformers/blob/0ffa22f9f6662ec9a0b6b6225bf152d32ab3e151/src/transformers/utils/generic.py#L392)), any name can be given to downstream classes. 

In terms of code, `model_name.startswith("TF")` would become `"transformers.modeling_tf_utils.TFPreTrainedModel" in str(inspect.getmro(model_class))`.

However, to ensure correctness, we would need to trigger a series of changes, as several places in our codebase rely on class names prefixed with the framework. Since this is my first time seeing this problem, I'll err toward no change. Nevertheless, I wanted to leave this comment here for future reference :)

cc @sgugger <|||||>I think it would be great to use a class check instead, maybe checking if we inherit from a keras model or an nn module?