Sorry, this does not really make sense to me.
Your models i>0 have different weights than the weights for the one with i=0. So the gradients of the i>0 models wrt w_0 are 0.
Given your torch.no_grad, I guess that you don’t plan on backpropagating through the weight updates.