compute perplexity using a custom metric function
Hello,

I am trying to replicate the "On the Cross-lingual Transferability of Monolingual Representations" paper from Artetxe et al. and I am using the code you're providing, specifically [run_mlm.py]
(https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_mlm.py) 

I wanted to log the perplexity to tensorboard during the evaluation step. I found out that the best option is to add a custom compute_metrics function in the trainer that uses the evaluation results (predictions and target) to compute perplexity. However, I didn't manage to do that because I couldn't understand what the output of predictions represents.

I am really new to NLP and your help is very much appreciated.
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?

cc @sgugger 

Thanks!<|||||>The predictions are the logits of your model, so in the case of a language model, it will be a big array `num_samples x seq_length x vocab_size`. Your labels will be a big array of `num_samples x seq_length` with the tokens corresponding to something not masked at -100 (index that is ignored).<|||||>First, I would like to apologize for writing to you here, I wasn't aware that there is a dedicated forum.
Thank you @sgugger, I appreciate your help, my function is working now.