Model performance graph is oscillating? I've seen a fair amount of information online regarding the loss graph of a deep learning model oscillating, but not so much about the performance graph. In general, when I "properly" train a neural network, the performance usually increase and then plateaus. However, I've noticed that in some cases the performance itself is very shaky. What is the cause of such behavior and what could be a fix?
One thing that comes to mind is that decreasing the learning rate may help because the variance seems high, but I'm not sure if this is based on any proof. Any tips are appreciated. Thanks.
Edit
Here's an example of what I'm talking about. I would expect that the macro-F1 score should also display similar patterns to micro-F1 despite being lower. However, it's oscillating quite a bit.


With a classification problem one can compute multiple $F_1$ scores for the classes seperately. The micro and macro $F_1$ scores compute a single score for all classes together.
If a few categories have little representation in the validation sample then their individual $F_1$ scores will be noisy and this influences the grouped macro and micro $F_1$ scores. The differences in the weight these two give to different classes makes that the one can be more noisy than the other.

*

*The macro-$F_1$ score computes a non-weighted average of $F_1$ scores for multiple categories.
The noise in the $F_1$ score of one or more small categories influences the macro-f1 score as well.


*The micro-$F_1$ score sums all the false/true negative/positive cases before computing a single $F_1$ score.
The number of false/true negative/positive cases in the categories that are small will be small and so these have relatively less influence on the sum. The micro-$F_1$ score will be relatively less noisy in comparison to the macro-$F_1$ score which gives more weight to the smaller categories.
