To update, my model (Module) was wrapped in another class with functions like get_loss which called the forward function. I didn’t realize DataParallel requires outputs of forward to be tensors, so I had to write a higher-level wrapper which can handle use of the model class when it’s being parallelized and when it’s not. So I solved this problem.