Hi there,
I’m experiencing an unexpected behaviour working on a summarization model:
As I increase my training samples, as expected, my validation loss decreases BUT the ROUGE metrics do not improve (do not increase), so the best model based on validation loss is not the best model based on ROUGE, they are not even close: a model with a quite larger validation loss is the one with the best ROUGE scores.
Can anyone explaina bit this? How normal is it?
Thank you guys