Get the different result at BertModel
At old version pytorch-pretrained-bert :
I used the BertModel to fine-tuned, loss will decrease.

But I used the New version BertModel to use the same data to finetune, but loss won't decrease.


> optimzer 
>I have tried different optimzer AdamW ,BertAdam. 
 
> learning rate
>0.1 0.01 0.001 ... 0.0000000001
 
> Batch Size
>1

> input I put **input_ids** and **token_type_ids**
> At old version return _,CLS,I take CLS
> At new version return [seq_len,1,768],[seq_len,768],I take [0,1,:] or [0,:] loss won't decrease.

I don't know what the detail I miss in the new version?
 
Have you read in detail the [migration guide](https://github.com/huggingface/pytorch-transformers#migrating-from-pytorch-pretrained-bert-to-pytorch-transformers) of the readme?

There is also a new `run_glue` example which is an updated version of the previous `run_classifier` and that you can use as a starting point for designing your own fine-tuning scripts.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
