Is this a classification task? What loss are you using?
As the network progresses in training it has a tendency to get very sure of it’s predictions. It might predict something like 99.999999% instead of 99.7%. The increase in loss & accuracy at the same time might indicate that it is sooooo sure for its predictions that once it actually fucks something up it gets a really high loss.
I’ve concluded this myself so I’m not sure if it’s sound. Would also be interested in more input on the question