Dear @lucasval and @nielsr,
Thank you for your answers. In fact I already found what was wrong, I forgot to set the model in training mode at the end of my evaluation loop!
Doing so the dropout layers would only be random for the first 1/10th of each epoch, and would be in eval mode for the remaining 9/10th of each epoch, which explains the periodic behavior of the loss.
Again thank you for your replies!
Best regards,
Axel