Solved by Nikronic in post #2 
              
              
                If, you need to define your own custom dataset, you can load both images in __getitem__ method by using CustomDataset. Here is post for such a situation: Custom Dataset with some preprocessing 
Also, dataloading tutorial has complete info about problems with customized data. 
Bests