Hi All,
my question is very simple. Starting from a pre-trained (Italian) model, I fine-tuned it on a specific domain of interest, say X, using masked language model (MLM) training. I then computed perplexity on a test text on domain X and checked that the final model performs better than the pre-trained one.
Is it sufficient?
People who trained this language model Umberto using MLM training, performed several tests on downstream tasks such as NER and POS.
How did they do?
My understanding is that starting from Umberto, they fine-tune it on NER and test it on WikiNER-ITA for instance. Is there a simple procedure to do it?
Thanks!