Thanks for the detailing here.
How can we put the computation part of the losses on another gpu(GPU:4) rather than GPU:0 which happens by default.
It will be great to have the method as many networks are designed such that they cover ~11 gigs and this things holds you back while parallelizing such a network.