Solved by nutszebra in post #8 
              
              
                If you use DistributedDataSample, the total amount of samples seen in a single epoch is len(dataset). 
If you don’t use DistributedDataSampler, the total amount of samples seen in a single epoch is actually 4 times i.e., 4*len(dataset). However, we should count this single epoch as 4 epoch though.