Yes, I think you are right regarding the shuffling in DistributedSampler.
If you want to share arrays with a different shape you might want to check the shared_dict 7 implementation and see if this would be a valid approach.