Hello, in RoBERTa article, authors refer to the model’s perplexity. However, I have yet to find a clear definition of what perplexity means in the context of a model training on the Masked Language Modeling Objective as opposed to the Causal Language Modeling task.
Could someone give me a clear definition? Thanks!