Load Biobert pre-trained weights into Bert model with Pytorch bert hugging face run_classifier.py code
These are the steps I followed to get Biobert working with the existing Bert hugging face pytorch code.

1. I downloaded the pre-trained weights 'biobert_pubmed_pmc.tar.gz' from [the Releases page](https://github.com/naver/biobert-pretrained/releases).

2. I ran this command to convert the tf checkpoint to pytorch model

```
python pytorch-pretrained-BERT/pytorch_pretrained_bert/convert_tf_checkpoint_to_pytorch.py --tf_checkpoint_path="biobert/pubmed_pmc_470k/biobert_model.ckpt.index" --bert_config_file="biobert/pubmed_pmc_470k/bert_config.json" --pytorch_dump_path="biobert/pubmed_pmc_470k/Pytorch/biobert.model"
```

This created a file 'biobert.model' in the specified path.

3. As mentioned in this [link](https://modelzoo.co/model/pytorch-pretrained-bert) , I compressed 'biobert.model' created above and 'biobert/pubmed_pmc_470k/bert_config.json' together into a biobert_model.tar.gz

3. I then ran the run_classifier.py of hugging face bert with the following command, using the tar.gz created above.
```
python pytorch-pretrained-BERT/examples/run_classifier.py --data_dir="Data/" --bert_model="biobert_model.tar.gz" --task_name="qqp" --output_dir="OutputModels/Pretrained/" --do_train --do_eval --do_lower_case
```

I get the error 

```
'UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte' 
```

in the line 
```
tokenizer = BertTokenizer.from_pretrained(args.bert_model, do_lower_case=args.do_lower_case)
```

Am I doing something wrong?

I just wanted to run run_classifier.py code provided by hugging face with biobert pretrained weights in the same way that we run bert with it. Is there a way to do this?
Have you tried the solutions discussed in the other issues on this topic:
- https://github.com/huggingface/pytorch-pretrained-BERT/issues/312
- https://github.com/huggingface/pytorch-pretrained-BERT/issues/239<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>> Have you tried the solutions discussed in the other issues on this topic:
> 
> * #312
> * #239

Hi @thomwolf ,

I followed the instructions [here](https://github.com/BaderLab/saber/issues/135#issuecomment-494455933) to convert the checkpoint and then placing the files (pytorch_model.bin, bert_config.json, and vocab.txt) in one folder to compress it. I did not need to ignore any weights as mentioned in any solutions you mentioned above ( #312 or #239 )

I copied the compressed folder to the home folder of 'pytorch-transformers'. Then I ran the following command from there itself to run the example code (`'examples/run_glue.py'`) on my data.

Then, I am trying to run the example code given by running the following command,

`python ./examples/run_glue.py \ --model_type bert \ --model_name_or_path biobert.gz \ --task_name=sts-b \ --do_train \ --do_eval \ --do_lower_case \ --data_dir=$DIR \ --max_seq_length 128 \ --per_gpu_eval_batch_size=8 \ --per_gpu_train_batch_size=8`

But, I get the same error as mentioned in the main discussion:

`UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte`

at this location:

`File "./examples/run_glue.py", line 424, in main config = config_class.from_pretrained(args.config_name if args.config_name else args.model_name_or_path, num_labels=num_labels, finetuning_task=args.task_name)`

Can you please tell what to change.<|||||>Try to pass the extracted folder of your converted bioBERT model to the `--model_name_or_path` :)

Here's a short example:

* Download the *BioBERT v1.1 (+ PubMed 1M)* model (or any other model) from the [bioBERT repo](https://github.com/naver/biobert-pretrained)
* Extract the downloaded file, e.g. with `tar -xzf biobert_v1.1_pubmed.tar.gz`
* Convert the bioBERT model TensorFlow checkpoint to a PyTorch and PyTorch-Transformers compatible one: `pytorch_transformers bert biobert_v1.1_pubmed/model.ckpt-1000000 biobert_v1.1_pubmed/bert_config.json biobert_v1.1_pubmed/pytorch_model.bin`
* Move config `mv biobert_v1.1_pubmed/bert_config.json biobert_v1.1_pubmed/config.json`

Then pass the folder name to the `--model_name_or_path` argument. You can run this simple script to check, if everything works:

```python
from pytorch_transformers import BertModel
model = BertModel.from_pretrained('biobert_v1.1_pubmed')
```<|||||>How we load a tensor as a pretrained model in bert<|||||>@stefan-it As per new `transformers-cli` third command would change as follows:

```bat
transformers-cli convert --model_type bert \
--tf_checkpoint biobert_v1.1_pubmed/model.ckpt-1000000 \
--config biobert_v1.1_pubmed/bert_config.json \
--pytorch_dump_output biobert_v1.1_pubmed/pytorch_model.bin
```<|||||>Hello!
Just to complement the @stefan-it instructions in step number 3, it works for me the following code:

`import os`
`from pytorch_pretrained_bert.convert_tf_checkpoint_to_pytorch import convert_tf_checkpoint_to_pytorch`
`path_bin = 'my_directory/pytorch_model.bin' `
`path_bert = 'my_bert_directory/' `

`if (not os.path.exists(path_bin)): `
`	convert_tf_checkpoint_to_pytorch( `
`		path_bert + "biobert_model.ckpt", `
`		path_bert + "bert_config.json",  `
`		path_bert + "pytorch_model.bin" `
`		)`

My folder (biobert_v1.0_pmc) was originally 5 files:
3 Tensorflow checkpoint files
A vocab file
A config file
<|||||>> * Move config `mv biobert_v1.1_pubmed/bert_config.json biobert_v1.1_pubmed/config.json`

Tnank you very much! It did help me!
<|||||>Thank you every one, this works fine now!<|||||>Dear all,
Thank you very much for the suggestions on how to prepare the model to be used with hugginfaces. I am trying to use huggingfaces BertTokenizer to perform NER on biomedical data with the pre-trained weights. 
It seems to work fine until then point when I want to map annotated tokens to entity labels. I have token ids and prediction ids, but I cannot figure out how/where to get label_list to follow an example of mapping from https://huggingface.co/transformers/usage.html#named-entity-recognition  
Thank you very much for any help you can provide!
Maria<|||||>Dear all,
I am a newbie and I don not have so much experience. Does anyone have a full tutorial or code for a regression task, Please share with me! I would greatly appreciate it! Thank you.<|||||>@stefan-it @nipunsadvilkar Thank you for your solutions.