When A is a module with multiple sub-modules, where only a sub-set of the sub-modules are used dependent on the input, for example, A is defined as follows,
class A(nn.Module):
    def __init__(self):
        self.common = nn.Linear(100, 50)
        self.module1 = nn.Linear(50,30)
        self.module2 = nn.Linear(50,2)
    def forward(self, input, idx):
        commonOut = self.common(input)
        if idx == 0:
            return self.module1(commonOut)
        else:
            return self.module2(commonOut)

opt = torch.optim.Adam(A.parameters()) 

In this case, if A is forwarded with idx=0, then the parameters belong to the module2 is not necessary to be updated.
When using opt for the parameter update, I wonder if PyTorch automatically only updates the modules that are acutally used in forward or the unused modules are also updated by adding zero gradient, which would be waste of resource.
If it does, is there any way I can get the list of parameters that are actually updated?
Thanks