hey @CDobbs



 CDobbs:

I’ve created a custom compute_metrics function that calculates f1 score for each entity and the macro-average. However, this doesn’t help steer the model. Do I need to create a custom loss function?


if you’re using the Trainer for fine-tuning, then you shouldn’t need a custom loss function. speaking of losses, does the train / validation loss decrease during fine-tuning?
without seeing more details / code on your approach, it’s hard to debug the problem - can you share them?
ps. you might find it instructive to see the “official” tutorial on NER which contains many more details than the one you linked to: https://colab.research.google.com/github/huggingface/notebooks/blob/master/examples/token_classification.ipynb#scrollTo=n9qywopnIrJH 17