Solved by AladdinPerzon in post #15 
              
              
                Hopefully you get it to work now, look at my edited code I changed and used F.relu instead as it might be more clear to you. Using nn.ReLU() I’m only creating the module and using it multiple times in the forward() part.