NaN Person/Spearman corr. when fine-tuning BERT with example code and commands
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.5.1
- Platform: Google COLAB and Ubuntu 18 LTS (tested both)
- Python version: 3.7 and 3.6
- PyTorch version (GPU?): 1.8.1 with 1 GPU
- Tensorflow version (GPU?):
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @Rocketknight1

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 --> @sgugger, @patil-suraj

## Information

Model I am using (Bert, XLNet ...): `bert-base-cased`

The problem arises when using:
* [x] the official example scripts: transformers/examples/pytorch/text-classification/run_glue_no_trainer.py
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: STS-B 
* [ ] my own task or dataset: (give details below)


Here is the training log I got with the command for `run_glue_no_trainer.py` given in [README](https://github.com/huggingface/transformers/tree/master/examples/pytorch/text-classification#pytorch-version-no-trainer)
```
2021-05-03 00:56:13.263483: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
05/03/2021 00:56:14 - INFO - __main__ -   Distributed environment: NO
Num processes: 1
Process index: 0
Local process index: 0
Device: cuda
Use FP16 precision: False

Downloading and preparing dataset glue/stsb (download: 784.05 KiB, generated: 1.09 MiB, post-processed: Unknown size, total: 1.86 MiB) to /root/.cache/huggingface/datasets/glue/stsb/1.0.0/dacbe3125aa31d7f70367a07a8a9e72a5a0bfeb5fc42e75c9db75b96da6053ad...
Downloading: 100% 803k/803k [00:00<00:00, 993kB/s]
Dataset glue downloaded and prepared to /root/.cache/huggingface/datasets/glue/stsb/1.0.0/dacbe3125aa31d7f70367a07a8a9e72a5a0bfeb5fc42e75c9db75b96da6053ad. Subsequent calls will reuse this data.
loading configuration file https://huggingface.co/bert-base-cased/resolve/main/config.json from cache at /root/.cache/huggingface/transformers/a803e0468a8fe090683bdc453f4fac622804f49de86d7cecaee92365d4a0f829.a64a22196690e0e82ead56f388a3ef3a50de93335926ccfa20610217db589307
Model config BertConfig {
  "architectures": [
    "BertForMaskedLM"
  ],
  "attention_probs_dropout_prob": 0.1,
  "finetuning_task": "stsb",
  "gradient_checkpointing": false,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "id2label": {
    "0": "LABEL_0"
  },
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "label2id": {
    "LABEL_0": 0
  },
  "layer_norm_eps": 1e-12,
  "max_position_embeddings": 512,
  "model_type": "bert",
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "pad_token_id": 0,
  "position_embedding_type": "absolute",
  "transformers_version": "4.5.1",
  "type_vocab_size": 2,
  "use_cache": true,
  "vocab_size": 28996
}

loading configuration file https://huggingface.co/bert-base-cased/resolve/main/config.json from cache at /root/.cache/huggingface/transformers/a803e0468a8fe090683bdc453f4fac622804f49de86d7cecaee92365d4a0f829.a64a22196690e0e82ead56f388a3ef3a50de93335926ccfa20610217db589307
Model config BertConfig {
  "architectures": [
    "BertForMaskedLM"
  ],
  "attention_probs_dropout_prob": 0.1,
  "gradient_checkpointing": false,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "layer_norm_eps": 1e-12,
  "max_position_embeddings": 512,
  "model_type": "bert",
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "pad_token_id": 0,
  "position_embedding_type": "absolute",
  "transformers_version": "4.5.1",
  "type_vocab_size": 2,
  "use_cache": true,
  "vocab_size": 28996
}

loading file https://huggingface.co/bert-base-cased/resolve/main/vocab.txt from cache at /root/.cache/huggingface/transformers/6508e60ab3c1200bffa26c95f4b58ac6b6d95fba4db1f195f632fa3cd7bc64cc.437aa611e89f6fc6675a049d2b5545390adbc617e7d655286421c191d2be2791
loading file https://huggingface.co/bert-base-cased/resolve/main/tokenizer.json from cache at /root/.cache/huggingface/transformers/226a307193a9f4344264cdc76a12988448a25345ba172f2c7421f3b6810fddad.3dab63143af66769bbb35e3811f75f7e16b2320e12b7935e216bd6159ce6d9a6
loading file https://huggingface.co/bert-base-cased/resolve/main/added_tokens.json from cache at None
loading file https://huggingface.co/bert-base-cased/resolve/main/special_tokens_map.json from cache at None
loading file https://huggingface.co/bert-base-cased/resolve/main/tokenizer_config.json from cache at /root/.cache/huggingface/transformers/ec84e86ee39bfe112543192cf981deebf7e6cbe8c91b8f7f8f63c9be44366158.ec5c189f89475aac7d8cbd243960a0655cfadc3d0474da8ff2ed0bf1699c2a5f
loading weights file https://huggingface.co/bert-base-cased/resolve/main/pytorch_model.bin from cache at /root/.cache/huggingface/transformers/092cc582560fc3833e556b3f833695c26343cb54b7e88cd02d40821462a74999.1f48cab6c959fc6c360d22bea39d06959e90f5b002e77e836d2da45464875cda
Some weights of the model checkpoint at bert-base-cased were not used when initializing BertForSequenceClassification: ['cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias']
- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-cased and are newly initialized: ['classifier.weight', 'classifier.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
100% 6/6 [00:00<00:00, 17.48ba/s]
100% 2/2 [00:00<00:00, 21.17ba/s]
100% 2/2 [00:00<00:00, 23.50ba/s]
05/03/2021 00:56:24 - INFO - __main__ -   Sample 468 of the training set: {'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'input_ids': [101, 138, 1825, 1110, 4518, 6870, 1105, 18700, 1113, 170, 2727, 1104, 19915, 119, 102, 138, 1685, 4648, 1110, 2807, 1113, 170, 3267, 119, 102], 'labels': 0.0, 'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}.
05/03/2021 00:56:24 - INFO - __main__ -   Sample 3376 of the training set: {'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'input_ids': [101, 1124, 8121, 1224, 1142, 1989, 1113, 1103, 1148, 1352, 3143, 1118, 170, 1646, 1697, 119, 102, 1828, 6096, 8121, 1113, 9667, 1113, 1103, 1148, 1352, 3143, 1118, 1126, 1237, 1697, 119, 102], 'labels': 3.0, 'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}.
05/03/2021 00:56:24 - INFO - __main__ -   Sample 3943 of the training set: {'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'input_ids': [101, 17448, 118, 21145, 10788, 1107, 1497, 17192, 3170, 102, 19553, 17448, 118, 21145, 4876, 3243, 1166, 1497, 17192, 3170, 102], 'labels': 3.799999952316284, 'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}.
05/03/2021 00:56:28 - INFO - __main__ -   ***** Running training *****
05/03/2021 00:56:28 - INFO - __main__ -     Num examples = 5749
05/03/2021 00:56:28 - INFO - __main__ -     Num Epochs = 3
05/03/2021 00:56:28 - INFO - __main__ -     Instantaneous batch size per device = 32
05/03/2021 00:56:28 - INFO - __main__ -     Total train batch size (w. parallel, distributed & accumulation) = 32
05/03/2021 00:56:28 - INFO - __main__ -     Gradient Accumulation steps = 1
05/03/2021 00:56:28 - INFO - __main__ -     Total optimization steps = 540
 33% 179/540 [00:24<00:52,  6.93it/s]/usr/local/lib/python3.7/dist-packages/scipy/stats/stats.py:3508: PearsonRConstantInputWarning: An input array is constant; the correlation coefficent is not defined.
  warnings.warn(PearsonRConstantInputWarning())
/usr/local/lib/python3.7/dist-packages/numpy/lib/function_base.py:2559: RuntimeWarning: invalid value encountered in true_divide
  c /= stddev[:, None]
/usr/local/lib/python3.7/dist-packages/numpy/lib/function_base.py:2560: RuntimeWarning: invalid value encountered in true_divide
  c /= stddev[None, :]
05/03/2021 00:56:55 - INFO - /usr/local/lib/python3.7/dist-packages/datasets/metric.py -   Removing /root/.cache/huggingface/metrics/glue/stsb/default_experiment-1-0.arrow
05/03/2021 00:56:55 - INFO - __main__ -   epoch 0: {'pearson': nan, 'spearmanr': nan}
 66% 359/540 [00:51<00:25,  7.12it/s]05/03/2021 00:57:22 - INFO - /usr/local/lib/python3.7/dist-packages/datasets/metric.py -   Removing /root/.cache/huggingface/metrics/glue/stsb/default_experiment-1-0.arrow
05/03/2021 00:57:22 - INFO - __main__ -   epoch 1: {'pearson': nan, 'spearmanr': nan}
100% 539/540 [01:19<00:00,  7.31it/s]05/03/2021 00:57:50 - INFO - /usr/local/lib/python3.7/dist-packages/datasets/metric.py -   Removing /root/.cache/huggingface/metrics/glue/stsb/default_experiment-1-0.arrow
05/03/2021 00:57:50 - INFO - __main__ -   epoch 2: {'pearson': nan, 'spearmanr': nan}
Configuration saved in /tmp/stsb/config.json
Model weights saved in /tmp/stsb/pytorch_model.bin
100% 540/540 [01:22<00:00,  6.51it/s]
```


## To reproduce

Steps to reproduce the behavior:

1. `git clone https://github.com/huggingface/transformers.git`
2. `pip install -r transformers/examples/pytorch/text-classification/requirements.txt`
3. `pip install transformers`
4. Execute the command given in [README](https://github.com/huggingface/transformers/tree/master/examples/pytorch/text-classification#pytorch-version-no-trainer)
```
mkdir /tmp/stsb/ -p
python transformers/examples/pytorch/text-classification/run_glue_no_trainer.py \
  --model_name_or_path bert-base-cased \
  --task_name stsb \
  --max_length 128 \
  --per_device_train_batch_size 32 \
  --learning_rate 2e-5 \
  --num_train_epochs 3 \
  --output_dir /tmp/stsb/
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
Seeing about 88.64 and 88.48 for Person and Spearman correlations respectively, but I consistently observed through multiple runs that both values are NaN
<!-- A clear and concise description of what you would expect to happen. -->

Oh yes, we forgot to add special case for sts-b (which is a regression task) in this script. Will send a fix later this morning! Thanks for flagging!<|||||>Thank you @sgugger for the prompt fix!