Thanks for the information you provided. Unfortunately, I no longer have access to the infrastructure where I ran the experiments, so I cannot see what happens with DDP. However, do you have an idea why nn.DataParallel may be causing the differences in CPU usage?