Thanks a lot for all these explanations. Except for transformation in dataloader, I was able to apply the rest of the feedback and seems it is actually working (I am not 100% sure). I did a minimal test with 10 images and two classes with 2 epochs.
Here, I expect each batch (say batch 1) to appear 9 times in the results which does.
Here is the new code: https://pastebin.com/MECLdfFZ 55
and here is the output: https://pastebin.com/SKQNRQNa 9
Can you help a little further how to separate the data loader for train and the test in loocv? the one in original tutorial was kind of very different and I am not sure how to make it work in this case.