No, you just can calculate the loss etc. as usual.
You would just need to make sure the tensors and parameters are on the appropriate device.
In the example code you could just call w.backward() or calculate the loss with a target on GPU1 and call loss.backward().