If it is set to sum, then you will get different scales of loss,
L = sum of all(losses)*batch_size/N.
With lower batch_size for validation, your loss will be lesser than train.
To correct this, you can either divide each batch loss with batch size or equivalently perform loss.mean(). This should give you proper values.