Hi, I think there is a simper way by combining ‘sign’ and ‘relu’:
# test input
input = torch.tensor([-20, 11], requires_grad=True).float()

# binary
sign = torch.sign(input)
binary_out = torch.relu(sign)
print(binary_out) # 0., 1.


Theoretically, sign (or unit-step) is not differentiable, but it can be backpropagated in PyTorch.
I have no idea how they design it…
Here is a question about this:

  
    
    
    Circumventing an indifferentiablility problem autograd
  
  
    Sorry if my question isn’t appropriate to ask here! It’s a bit theory related question. But I want people to share knowledge to understand what’s happening in neural networks. 
As far as I understand, Pytorch use chain rule to compute gradients of loss w.r.t. network parameters. 
Therefore, when we use an indifferentiable function such as step function (torch.sign() ) in the neural network, the gradient won’t be propagated hence loss won’t decrease. 
In the code below, I implemented a very simpl…