So, to clarify, this should mean that if my loss function for the policy network were just loss = q1(state, policy_output), where policy_output was calculated on gpu but q1 was on cpu, i would move the policy_output to cpu (this would be required for it to function I think?), and then a call of loss.backward() would correctly backprop through the policy network?
My thanks for your help