Solved by dejanbatanjac in post #3 
              
              
                Yes, you should use sigmoid function. 
def sigmoid(x): return 1/(1 + (-x).exp())

It will convert the space of [-inf, inf] into a probability [0,1]. 
Note this sigmoid works on a tensor. So it will do that for all your activations. 
What ever goes to the sigmoid you can call “logit”, even though thi…