You might find the recent discussions on composable datasets useful or interesting.

  
      github.com/pytorch/pytorch
  
  
    
  
	  
  

  
    
      [RFC] DataLoader architecture updates and TarDataset implementation 28
    

    
      
        opened 
        
          
        
        Dec 16, 2020
      
      


      
        
          
          VitalyFedyunin
        
      
    
  



  Problem statement
This proposal aims to construct a modular, user-friendly, and performant toolset to address the ambiguous activity referred to as “dataloading”...



    feature
    module: dataloader
    triaged