Solved by achaiah in post #7 
              
              
                FYI, just to update this topic in case someone else is looking for a good answer, there is now a ConcatDataset in Pytorch that does pretty much what the author was looking for.