Make sure you mount the data on the same device as that of the model. When you load the model use,  torch.nn.DataParallel(model, device_ids=[0, 1,2,..]). The first device(in the above example, device 0 will be where the model is mounted before it is replicated).
Make sure you mount input on this device itself. The first gpu will be used as a platform to scatter it onto other devices.