Thanks Alan! I was wondering how do I send all the tensors to one gpu, and perform operations, which I want to perform, before the value of any of the tensor changes in any of their respective gpus.
Meaning, I don’t want the values of these tensor present on different gpus to change, before I complete my operation and send them back to respective gpu.