Hi, @lycaenidae. Some questions/comments, but fair warning… I’m not very experienced with pytorch. 

What is the scale of your plot?
Are you calculating loss as an instantaneous value? A moving average?
What is your use case? Without some context, it’s tough to tell what could be going on.
Are you shuffling the samples in your training set? If not, your instantaneous loss values could definitely jump around like that. Datasets are sometimes “clumpy”. Imagine hitting your model with a large “clump” of a novel class after you’ve been training for a while.