Hi,
I am working on a multi-label topic classifier that can classify webpages into some of our ~100 topics. The classifier currently uses a basic Neural Network and I wish to adapt the XLM-R model provided by Huggingface to give the classifier multi-lingual capabilities.
However, when I train a classifier using XLM-R the performance (using pr_auc) is worse than that of the classifier using the basic Neural Network.
What can I do to improve the performance of a transformer-based Neural Network model?