<p>Have a look at <a href="https://medium.com/the-artificial-impostor/news-topic-similarity-measure-using-pretrained-bert-model-1dbfe6a66f1d" rel="nofollow noreferrer">https://medium.com/the-artificial-impostor/news-topic-similarity-measure-using-pretrained-bert-model-1dbfe6a66f1d</a></p>

<p>You can have the two sentences as first and second use the next sentence score as a similarity measure. You can further fine-tune your model on some semantic similarity tasks like Sent-Eval or your own dataset if you have one</p>

CC BY-SA 4.0
2019-08-12T18:13:56.810Z
2019-08-12T18:13:56.810Z
cookiemonster
<p>I am unaware to use the derived checkpoints from pre-trained BERT model for the task of semantic text similarity.</p>

<pre><code>!python create_pretraining_data.py \
          --input_file=/input_path/input_file.txt \
          --output_file=/tf_path/tf_examples.tfrecord \
          --vocab_file=/vocab_path/uncased_L-12_H-768_A-12/vocab.txt \
          --do_lower_case=True \
          --max_seq_length=128 \
          --max_predictions_per_seq=20 \
          --masked_lm_prob=0.15 \
          --random_seed=12345 \
          --dupe_factor=5

!python run_pretraining.py \
      --input_file=/tf_path/tf_examples.tfrecord \
      --output_dir=pretraining_output \
      --do_train=True \
      --do_eval=True \
      --bert_config_file=/bert_path/uncased_L-12_H-768_A-12/bert_config.json \
      --init_checkpoint=/bert_path/uncased_L-12_H-768_A-12/bert_model.ckpt\
      --train_batch_size=32 \
      --max_seq_length=128 \
      --max_predictions_per_seq=20 \
      --num_train_steps=20 \
      --num_warmup_steps=10 \
      --learning_rate=2e-5
</code></pre>

<p>I have run a pre-trained BERT model with some domain of corpora from scratch. I have got the checkpoints and graph.pbtxt file from the code above. But I am unaware on how to use those files for evaluating semantic text similarity test file.</p>

<natural-language-processing><bert>
How to use pretrained checkpoints of BERT model on semantic text similarity task?
gsasikiran
# How to use pretrained checkpoints of BERT model on semantic text similarity task?

(Asked by: username_0 on 2019-08-12)

I am unaware to use the derived checkpoints from pre-trained BERT model for the task of semantic text similarity.
!python create_pretraining_data.py \
          --input_file=/input_path/input_file.txt \
          --output_file=/tf_path/tf_examples.tfrecord \
          --vocab_file=/vocab_path/uncased_L-12_H-768_A-12/vocab.txt \
          --do_lower_case=True \
          --max_seq_length=128 \
          --max_predictions_per_seq=20 \
          --masked_lm_prob=0.15 \
          --random_seed=12345 \
          --dupe_factor=5

!python run_pretraining.py \
      --input_file=/tf_path/tf_examples.tfrecord \
      --output_dir=pretraining_output \
      --do_train=True \
      --do_eval=True \
      --bert_config_file=/bert_path/uncased_L-12_H-768_A-12/bert_config.json \
      --init_checkpoint=/bert_path/uncased_L-12_H-768_A-12/bert_model.ckpt\
      --train_batch_size=32 \
      --max_seq_length=128 \
      --max_predictions_per_seq=20 \
      --num_train_steps=20 \
      --num_warmup_steps=10 \
      --learning_rate=2e-5

I have run a pre-trained BERT model with some domain of corpora from scratch. I have got the checkpoints and graph.pbtxt file from the code above. But I am unaware on how to use those files for evaluating semantic text similarity test file.

**Tags:** natural-language-processing, bert


---

## 1 answers

### Answer by username_1 on 2019-08-12. Score: 1

Have a look at <https://medium.com/the-artificial-impostor/news-topic-similarity-measure-using-pretrained-bert-model-1dbfe6a66f1d>

You can have the two sentences as first and second use the next sentence score as a similarity measure. You can further fine-tune your model on some semantic similarity tasks like Sent-Eval or your own dataset if you have one


---

