Among the different test I have done, one seems stupid but may reveals something strange. I have used as Test/Validation set the same samples done used for training except that I do not flip H or V nor rotate by 90deg,180deg or 270deg, here is the test & train losses: