So I found that the ImageFolder sorts the file names. Hence I made sure that my names of the rgb and depth images in the folder are corresponding and same. Now I pick the same rgb and depth image from the folder using ImageFolder.