Why not use
binarize = lambda x: F.relu(x) * 2 - 1

The problem might be that you modify the input inplace without telling pytorch that the input is dirty.  In forward you need to do this…
self.mark_dirty(input)

For more info, see here…

  
    
    
    Understand mark_dirty() 
  
  
    So I read the inline documentation about mark_dirty() here: 


I don’t quite understand what extra checks are needed for inplace operators. Would be great if the devs can give some hints. Thanks!