Solved by albanD in post #4 
              
              
                Ho right it is, my bad. 
It is a bit tricky for Siamese network as you need to accumulate the gradients for all tree runs. 
One simple way to do this is to make three copies of your network, one on each device, then send each copy to it’s respective device. 
After each backward, you will need to acc…