If both w and z are used to compute y and x_i, you will need to do two backward passes. One to get the gradients wrt to w and one to get the gradients wrt to z. You can either use the autograd.grad method to state explicitly the gradient of which element you want or simply save the .grad field of the variables you care about and ignore the other ones.