I’m not sure I understand the question completely, but during the forward pass the operations applied on the tensor will be tracked by Autograd and the gradient will be calculated using the computation graph. It doesn’t matter where the parameter is registered as its usage will be tracked.