Solved by albanD in post #10 
              
              
                Hi, 
Yes this comment was more to explain the details of the issue without writing too long stuff on the github PR. 
You can either use inplace=False or use the version of relu that I gave you above with inplace=True.