If you have learnable parameters/dynamic buffers specific to your classes or some tensors initialized statically inside your classes (eg., self.tensor = torch.arange()), make sure to register_buffer() 1 or register_parameter() 1 (depending on your need), so that DataParallel knows what to copy to other GPUs.