Thanks for your answers, it’s very helpful in understanding how things work.
Just wondering one more thing – I’m still getting some weird numbers and noticed the code I’m using has a different batch size for the dataloader for training data (batchsize = 2) and for the validation data (batchsize = 1).
Since the loss is currently loss/len(dataloader), would that be the source of why my validation losses are significantly below my training losses? What would be the correct way of recording the losses per epoch for both the validation and the training if the batch sizes are different?