Solved by ptrblck in post #2 
              
              
                Aurelien’s explanations seem reasonable and it’s what I observe usually. 
If you want to make sure dropout and the running estimates might be the cause for the gap, you could set your model to model.eval() and use the training dataset after the epoch (same as where you are calculating the validation…