Solved by KFrank in post #2 
              
              
                Hi Magjo! 

I’m guessing …   (But it’s the weekend, so let me guess away!) 
At issue is not only whether you are using an inplace operation, 
but also whether the tensor modified by the inplace operation is 
used directly in the backward() computation. 
The derivative of relu() depends on the sign o…