Hi …
about the data tensor, you also need to send to device, then if it’s a matrix the pytoch will manage to split the matrix in a half for each GPU.
Example:
https://pytorch.org/tutorials/beginner/blitz/data_parallel_tutorial.html#sphx-glr-beginner-blitz-data-parallel-tutorial-py 44
But if you are trying to control each GPU individually then you need to create a sub model of each set of GPU and just send the information to the specific device