Hi
Yes, I found a way but not sure whether it’s the best way.
What I’m doing is for each sentence I’m masking a random word in the sentence and ask the model to predict it. If the actual word is in the predicted list I’m increasing the tp(positive) value or else the tn(negative) value. finally I’m calculating the accuracy (tp/tp+tn).
and I’m calculating the perplexity value using the following code
  
      

      colab.research.google.com
  

  
    

Google Colaboratory 20



  

  
    
    
  

  


Hope my answer is clear to you…