Solved by smth in post #2 
              
              
                There isn’t an automatic way to do this. If A is a list of Tensors, each on a separate GPU, I presume A is a large matrix, with rows 0 to i on GPU0, i to j on GPU1, etc. 
Let’s assume B is only on GPU 0, because you didn’t mention anything about B. 
Here’s a sample snippet showing how to parallelize…