Model.eval() always returns the same logits for SequenceClassfication models with binary labels
### System Info

On GoogleColab
- `transformers` version: 4.27.4
- Platform: Linux-5.10.147+-x86_64-with-glibc2.31
- Python version: 3.9.16
- Huggingface_hub version: 0.13.3
- PyTorch version (GPU?): 2.0.0+cu118 (True)
- Tensorflow version (GPU?): 2.12.0 (True)
- Flax version (CPU?/GPU?/TPU?): 0.6.8 (gpu)
- Jax version: 0.4.7
- JaxLib version: 0.4.7
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: not

### Who can help?

@younesbelkada @sgugger

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from sklearn.model_selection import train_test_split
import numpy as np
import torch
from torch.utils.data import Dataset, DataLoader
from evaluate import load
from tqdm import tqdm
from transformers import AutoTokenizer, AutoModelForSequenceClassification
from torch import cuda
from datasets import load_dataset
from transformers import TrainingArguments, Trainer

device = 'cuda' if cuda.is_available() else 'cpu'

tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")
model = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased", num_labels=2)

dataset = load_dataset("ethos", "binary")

def tokenize_function(examples):

    return tokenizer(examples["text"], max_length = 512, padding="max_length", truncation=True)


tokenized_datasets = dataset.map(tokenize_function, batched=True)

small_train_dataset = tokenized_datasets["train"].shuffle(seed=42).select(range(0, 898))

small_eval_dataset = tokenized_datasets["train"].shuffle(seed=42).select(range(898, 998))

load_accuracy = load("accuracy")
load_f1 = load("f1")

def compute_metrics(eval_pred):  
   logits, labels = eval_pred
   predictions = np.argmax(logits, axis=-1)
   accuracy = load_accuracy.compute(predictions=predictions, references=labels)["accuracy"]
   f1 = load_f1.compute(predictions=predictions, references=labels)["f1"]
   return {"accuracy": accuracy, "f1": f1}

training_args = TrainingArguments(output_dir="test_trainer", evaluation_strategy="epoch", learning_rate = 1e-03, per_device_train_batch_size =16,
                                  per_device_eval_batch_size=4,
                                  num_train_epochs=1)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=small_train_dataset,
    eval_dataset=small_eval_dataset,
    compute_metrics=compute_metrics,
)

trainer.train()

### Expected behavior

```trainer.predict(small_eval_dataset)``` should return different logits but it returns the same for all the test examples
```
PredictionOutput(predictions=array([[ 0.30563816, -0.10853065],
       [ 0.30566448, -0.10852079],
       [ 0.3056038 , -0.10854296],
       [ 0.30563852, -0.10852969],
       [ 0.30562696, -0.10853519],
       [ 0.3057046 , -0.10850368],
       [ 0.3056232 , -0.10853792],
       [ 0.3056584 , -0.10852299],
       [ 0.30566052, -0.10852136],
       [ 0.30566704, -0.10851857],
       [ 0.30566064, -0.10852212],
       [ 0.30565894, -0.10852377],
       [ 0.30565098, -0.10852514],
       [ 0.30566713, -0.10852013],
....
````
```python
inputs = torch.tensor(small_eval_dataset['input_ids']).to(device)
mask = torch.tensor(small_eval_dataset['attention_mask']).to(device)

model.train()
model(inputs[0:10], mask[0:10])
```
```
SequenceClassifierOutput(loss=None, logits=tensor([[ 0.2427, -0.2602],
        [ 0.2804, -0.2819],
        [ 0.0620, -0.1497],
        [ 0.6520, -0.3421],
        [ 0.5095, -0.1113],
        [ 0.3538,  0.0181],
        [ 0.2826,  0.1292],
        [ 0.4033,  0.0041],
        [ 0.4308, -0.1813],
        [ 0.3979, -0.2117]], device='cuda:0', grad_fn=<AddmmBackward0>), hidden_states=None, attentions=None)
```

```python
model.eval()
model(inputs[0:10], mask[0:10])
```
```
SequenceClassifierOutput(loss=None, logits=tensor([[ 0.3056, -0.1085],
        [ 0.3057, -0.1085],
        [ 0.3056, -0.1085],
        [ 0.3056, -0.1085],
        [ 0.3056, -0.1085],
        [ 0.3057, -0.1085],
        [ 0.3056, -0.1085],
        [ 0.3057, -0.1085],
        [ 0.3057, -0.1085],
        [ 0.3057, -0.1085]], device='cuda:0', grad_fn=<AddmmBackward0>), hidden_states=None, attentions=None)
```
That's because your model training diverged. It has nothing to do with the Transformers library and is probably due to your very high learning rate. You should go on the [forums](https://discuss.huggingface.co/) if you need help debugging your trainings.<|||||>Yes thanks this was the reason