Thank you for your reply. I think this is a major advantage of PyTorch over other TF.
Then here comes the second question.
In the implementation of ResNet, the downsample variable is passed to BasicBlock while initialization (what I call “internal” signal). However, in my use case, the conditional is activated by some “external” signal like number of epochs, so how do I pass this external signal to forward pass.
There are two thoughts

Declare some global variables using global VAR and let forward function to access it.
Directly pass VAR through forward, which makes signature look like

def forward(self, x, VAR):
   if VAR:
      # do something

So which one is correct and preferred?