It depends what the underlying implementation does. If it wraps nn.DataParallel, you should see a speedup. If it just processes the examples serially then not. When you run this, do you see GPU utilization on all the GPUs you expect to be participating (e.g. with nvidia-smi)?