Hi,
in pytorch lightning, recently I found that if I use F.Dropout in forward step, even when I set mode to model.val, the dropout still work, then I realize I should replace it with nn.Dropout as module attribute. After that, everything performs normally.

Then my concern would be like, if I use F.relu in forward instead of nn.ReLU as attribute, can the model performs normally in backward, i.e. considering there is a relu or relu just work in forward?
and if I want to use twice nn.ReLu, should I define one self.relu=nn.ReLU and use self.relu in 2 different position or should I define 2, i.e. self.relu1 = nn.ReLU, self.relu2 = nn.ReLU

So basically I am not very clear how the backpropagation find which module( or function) it should consider.