So my entire code base is actually quite large, but here are some necessary details.
As I said, the module import error occurred after changing transformations.transforms to transformations.single_transforms. This module is only directly imported in transformations.__init__ where my transform_factory code resides.
The multiprocessing code is structured as follows. As usual, I have some relevant setup in the main function, where I spawn the main_process function.
In the main_process function, the code is structured as:

Init process group
Obtain transformations (through transform_factory - probably the module in question)
Create Datasets, Distributed samplers, and Dataloaders
Create distributed model, loss fns, optimizers, etc.
Initialize the trainer class
Run training

The last step - run training - is basically just a nested for loop where I run one round of training followed by one round of evaluation. Simplified example:
    def run(self):
        self.model.train()
        for epoch in range(0, self.num_epochs):
            if self.train_sampler is not None:
                self.train_sampler.set_epoch(epoch)
            for phase in ['train', 'val']:
                if phase == 'train':
                    self.model.train()
                    train_results = self.train_one_epoch(epoch)
                    print(train_results)
                else:
                    self.model.eval()
                    val_results = self.validate(epoch)
                    print(val_results)

In no part of the training do I reference or try to import from transformations.single_transforms. And as you said, even if I did, it shouldn’t matter because of the  python Bytecode.
Some other details that may help…

I’ve been using PyTorch for around 3 years now, mostly using DataParallel, and I’ve never encountered this issue before. I only switched to Distributed training a few weeks ago, and it’s my first time seeing this problem.
The training doesn’t bug out as soon as the change is made. In fact, it will finish its current phase of training (until the dataloader is finished iterating), then die during the transition from train --> evaluation or vice versa.

Finally, thanks for the suggestion. I do in fact checkpoint every epoch, so I can resume training. I just wanted to get to the bottom of this because I just can’t understand why the code would crash mid-training. I talked to one of my colleagues about this, and he said that he’s experienced something similar in distributed training. He also noted that changing the model structure will also cause the code to crash, but I haven’t checked this for myself.