Solved by ptrblck in post #4 
              
              
                Yes, I think you are right regarding the shuffling in DistributedSampler. 
If you want to share arrays with a different shape you might want to check the shared_dict implementation and see if this would be a valid approach.