Solved by claracurrier in post #7 
              
              
                @mrshenli in the course of building a barebones repro, I discovered the source of the error: I was passing something unpicklable into my Dataset instance (I was passing my model instead of the args by accident) such that it looked like this: 
def main(rank, args):
  dist.init_process_group(...)
  du…