BERT + CNN classifier doesn't work after migrating from 0.1.2 to 0.4.0
I used BERT in a very simple sentence classification task:
in `__init__` I have
```python3
self.bert = BertModel(config)
self.cnn_classifier = CNNClassifier(self.config.hidden_size, intent_cls_num)
```
and in forward it's just
```python3
encoded_layers, _ = self.bert(input_ids, token_type_ids, attention_mask, output_all_encoded_layers=False)
confidence_score = self.cnn_classifier(encoded_layers)
masked_lm_loss = loss_fct(confidence_score, ground_truth_labels)
```
This code works perfectly when I use 0.1.2 version, but in 0.4.0, it:
- always predicting the most common class when have a large training set
- cannot even learn a dataset with only 4 samples (fed in as one batch); can learn a single sample though

Why are these problems happening in 0.4.0? The only change in my code is that I changed `weight_decay_rate` to `weight_decay`...
I don't know...
If you can open-source a self contained example with data and code I can try to give it a deeper look.
Are you using `apex`? That's the main change in 0.4.0.<|||||>Hi Thomas! I've found the problem. I think It's because you modified your `from_pretrained` function and I'm still using a part of the `from_pretrained` function from version 0.1.2, which resulted in some compatibility issues. Thanks!