Thank you for the reply. The code provided is like (a) in the picture(F1 == loss1 and F2 == loss2), will it be same for (b)? if so, could you please tell me where and how the information of  grad has been saved and used in pytorch?