Problems converting TF BioBERT model to PyTorch
My goal is to convert and train on the [BioBERT pretrained checkpoints](https://github.com/naver/biobert-pretrained) in pytorch and train on the [SQuAD v2.0 Dataset](https://rajpurkar.github.io/SQuAD-explorer/).  

I have (seemingly) successfully transfered the checkpoint using the `./pytorch_pretrained_bert/convert_tf_checkpoint_to_pytorch.py` [script](https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/pytorch_pretrained_bert/convert_tf_checkpoint_to_pytorch.py). 

I loaded the converted checkpoint into the `run_squad.py` [example](https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/examples/run_squad.py). I also changed the tokenizer to use the vocab file found with the BioBERT model. At this point, I was able to train the model and observe the loss to decrease. 

My first issue appeared when trying to write the SQuAD predictions. `best_non_null_entry.start_logit` did not have a `start_logit` because the `best_non_null_entry` was `NoneType`. This error resembles the previous issue #207. I implemented the solution found and my code was able to run. 

My results from training have been the same or worse than a random model. Nearly all of the SQuAD predictions are the "empty" string text from the fix of #207. 

**I believe the original cause of the `NoneType` error for `best_non_null_entry` is the reason for the failure to predict anything.** 

Are there specs to obey when converting a TF pretrained BERT model?
What would cause the `NoneType` error for `best_non_null_entry`?
Any and all help is appreciated.
I have solved my issue. All my code was correctly written. The error was an corrupted/improperly saved model.bin file.<|||||>I'm trying to convert BioBert to Pytorch also, so just wondering if you could share a bit more details on how you are doing the conversion. Thanks!<|||||>First, I downloaded the BioBERT TF checkpoints [here](https://github.com/naver/biobert-pretrained). Each model (i.e. biobert_pmc) should have three `.ckpt` files, a `vocab.txt` file, and a `bert_config.json` file. 

Initially, I tried to use the command line interface `pytorch_pretrained_bert convert_tf_checkpoint_to_pytorch` using the `bert_config.json` and `.ckpt` files seen above. I ran into `AttributeError: 'Parameter' object has no attribute 'BERTAdam'`. I followed the solution [here](https://github.com/dmis-lab/biobert/issues/2). 

To do this, I copied the `convert_tf_checkpoint_to_pytorch.py` file and the `load_tf_weights_in_bert` function found in `modeling.py`. I then added the two lines seen in the [solution above](https://github.com/dmis-lab/biobert/issues/2) in my own version of the function and file. 

Given correct file paths, this worked to convert all three BioBERT checkpoints into pytorch `.bin` files.<|||||>@jwhite2a Thank you! This worked for me.