@vintagedeek,

(1) Pre-training means running a corpus through the BERT architecture where masked language modeling and next sentence prediction are used to derive weights. You can do this (a) from scratch with your own vocabulary and randomly initialized weights or (b) using the pre-trained BERT vocab/weights (so you are in effect “pre-training a pre-trained model.”
(2) fine tuning means adding a layer to the BERT architecture for some downstream task, such as classification.

seems fine to me
for better embeddings and similarity, you may want to check this:
  
      
      github.com
  
  
    

UKPLab/sentence-transformers 117


  Sentence Embeddings with BERT &amp; XLNet. Contribute to UKPLab/sentence-transformers development by creating an account on GitHub.

  
  
    
    
  
  


and this:
  
      
      github.com
  
  
    

JohnGiorgi/DeCLUTR 57


  The corresponding code from our paper "DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations". Do not hesitate to open an issue if you run into any trouble! - JohnGi...