`bert-base-uncased` works for CoLA, `bert-large-uncased` always predicts one class
I'm having an issue with CoLA where finetuning off of bert-large results in a model that only predicts one class. 

I make one change in configs to train the large model - I set `train_batch_size` to `16` for `bert-large-uncased`. 

These are the two training commands I use (missing do_lowercase, I know, but it's forced anyways):

```
python ./run_classifier.py \
    --task_name CoLA \
    --do_train \
    --data_dir ./data/cola/ \
    --bert_model bert-large-uncased \
    --output_dir ./out/cola-finetune-large-uncased/
```

```
python ./run_classifier.py \
    --task_name CoLA \
    --do_train \
    --data_dir ./data/cola/ \
    --bert_model bert-base-uncased \
    --output_dir ./out/cola-finetune-base-uncased/
```

---

Now here are the results I get with base vs large models:

Here are my results evaluating with plain bert-large-uncased
```
eval_loss = 0.6977422047745098
mcc = -0.05490997894843018
```

Here are the results with bert-base-uncased
```
eval_loss = 1.013142795273752
mcc = 0.02904813156816523
```

Now here are the results with fine-tuning on bert-base-uncased: 
```
eval_loss = 0.590644522203189
mcc = 0.5313406823271718
```

Pretty much reflects what the paper says, nice. But when I do the exact same process, training on bert-large-uncased and with a slightly smaller batch size (16 instead of 32) b/c of GPU memory limitations, I get these results:

```
eval_loss = 0.61876411058686
mcc = 0.0
```

Just to be clear, in these examples I only change the `bert_model` flag from `bert-large-uncased` to `bert-base-uncased` and change the batch size when training, no other changes at all.

I feel I must be doing something wrong. I'm using `run_classifier.py` from this repo. Any ideas what could be the problem?

I've read there's some instability with BERT-Large for small datasets such as CoLA, but surely it doesn't degenerate this much? If I understand correctly mcc = 0 indicates random guessing... Or is that actually the case, and I just need to run it more times and cross my fingers for a non-degenerate run?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
