Solved by albanD in post #2 
              
              
                Hi, 
nn.ReLU() is a layer, but it has not weights or bias. 
The two are exactly the same. 
The version as a nn.Module is convenient to be able to add it directly into a nn.Sequential() construct for example. 
The functional version is useful when you write a custom forward and you just want to apply…