@mrshenli in the course of building a barebones repro, I discovered the source of the error: I was passing something unpicklable into my Dataset instance (I was passing my model instead of the args by accident) such that it looked like this:
def main(rank, args):
  dist.init_process_group(...)
  dummy_train = []
  model = dummyModel(args)
  model.parallelize(rank)

  train_dataset = MyNLPDataset(dummy_train, args)  
  # replace args with something unpicklable to trigger error

  train_sampler = torch.utils.data.distributed.DistributedSampler(...)
  train_loader = torch.utils.data.DataLoader(train_dataset ...)
  
  for epoch in range(args.num_epochs):
    for training_ex in train_loader:
      output = model.update(training_ex)

Where the dataset class was:
class MyNLPDataset(torch.utils.data.Dataset):
  def __init__(self, examples, args):
    self.examples = examples
    self.args = args # <-- previously this was saving a copy of the model object

# ... overrided methods ...

I think this could’ve been easier if I had a better error message - your comment about something being unpicklable helped me narrow my search. The error ultimately didn’t involve the process group.
I’m now getting a new NCCL backend error that doesn’t affect GLOO, so I’m back to hunting for new issues. Thanks so much for your help!