Answers.Body: <p>Have a look at <a href="https://medium.com/the-artificial-impostor/news-topic-similarity-measure-using-pretrained-bert-model-1dbfe6a66f1d" rel="nofollow noreferrer">https://medium.com/the-artificial-impostor/news-topic-similarity-measure-using-pretrained-bert-model-1dbfe6a66f1d</a></p>
  
  <p>You can have the two sentences as first and second use the next sentence score as a similarity measure. You can further fine-tune your model on some semantic similarity tasks like Sent-Eval or your own dataset if you have one</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-12T18:13:56.810Z
Answers.LastActivityDate: 2019-08-12T18:13:56.810Z
Answers.UserDisplayName: cookiemonster
Body: <p>I am unaware to use the derived checkpoints from pre-trained BERT model for the task of semantic text similarity.</p>
  
  <pre><code>!python create_pretraining_data.py \
            --input_file=/input_path/input_file.txt \
            --output_file=/tf_path/tf_examples.tfrecord \
            --vocab_file=/vocab_path/uncased_L-12_H-768_A-12/vocab.txt \
            --do_lower_case=True \
            --max_seq_length=128 \
            --max_predictions_per_seq=20 \
            --masked_lm_prob=0.15 \
            --random_seed=12345 \
            --dupe_factor=5
  
  !python run_pretraining.py \
        --input_file=/tf_path/tf_examples.tfrecord \
        --output_dir=pretraining_output \
        --do_train=True \
        --do_eval=True \
        --bert_config_file=/bert_path/uncased_L-12_H-768_A-12/bert_config.json \
        --init_checkpoint=/bert_path/uncased_L-12_H-768_A-12/bert_model.ckpt\
        --train_batch_size=32 \
        --max_seq_length=128 \
        --max_predictions_per_seq=20 \
        --num_train_steps=20 \
        --num_warmup_steps=10 \
        --learning_rate=2e-5
  </code></pre>
  
  <p>I have run a pre-trained BERT model with some domain of corpora from scratch. I have got the checkpoints and graph.pbtxt file from the code above. But I am unaware on how to use those files for evaluating semantic text similarity test file.</p>
  
Tags: <natural-language-processing><bert>
Title: How to use pretrained checkpoints of BERT model on semantic text similarity task?
UserDisplayName: gsasikiran
ThreadText: # How to use pretrained checkpoints of BERT model on semantic text similarity task?
  
  (Asked by: username_0 on 2019-08-12)
  
  I am unaware to use the derived checkpoints from pre-trained BERT model for the task of semantic text similarity.
  
  ```
  !python create_pretraining_data.py \
            --input_file=/input_path/input_file.txt \
            --output_file=/tf_path/tf_examples.tfrecord \
            --vocab_file=/vocab_path/uncased_L-12_H-768_A-12/vocab.txt \
            --do_lower_case=True \
            --max_seq_length=128 \
            --max_predictions_per_seq=20 \
            --masked_lm_prob=0.15 \
            --random_seed=12345 \
            --dupe_factor=5
  
  !python run_pretraining.py \
        --input_file=/tf_path/tf_examples.tfrecord \
        --output_dir=pretraining_output \
        --do_train=True \
        --do_eval=True \
        --bert_config_file=/bert_path/uncased_L-12_H-768_A-12/bert_config.json \
        --init_checkpoint=/bert_path/uncased_L-12_H-768_A-12/bert_model.ckpt\
        --train_batch_size=32 \
        --max_seq_length=128 \
        --max_predictions_per_seq=20 \
        --num_train_steps=20 \
        --num_warmup_steps=10 \
        --learning_rate=2e-5
  
  ```
  
  I have run a pre-trained BERT model with some domain of corpora from scratch. I have got the checkpoints and graph.pbtxt file from the code above. But I am unaware on how to use those files for evaluating semantic text similarity test file.
  
  **Tags:** natural-language-processing, bert
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-08-12. Score: 1
  
  Have a look at <https://medium.com/the-artificial-impostor/news-topic-similarity-measure-using-pretrained-bert-model-1dbfe6a66f1d>
  
  You can have the two sentences as first and second use the next sentence score as a similarity measure. You can further fine-tune your model on some semantic similarity tasks like Sent-Eval or your own dataset if you have one
  
  
  ---
  
  