Hi Richard!
It looks like this is related to your previous post:

  
    
    
    How to use sigmoid on only positive values? 
  
  
    I have a binary classification model, that in the latest linear layer, it outputs only positive values (don’t ask why, that’s a different matter), now when i give the final layer’s output to torch.sigmoid, all the results are above 50%, because the final linear layer is only outputting positive values, how can i fix this and output probability? is there any “positive only” sigmoid in pytorch?
  





 Richard_S:

I am trying to constraint the final layer of my NN to have non negative weights in the final layer, for my binary classification task ( the reason for me wanting to have non negative weights does not matter right now)


My guess is that you’re not going about your problem in a sensible way.
But unless you describe your actual use case, it’s hard to know.


        self.classify.weight.data = self.classify.weight.data.clamp(min=0)

is this proper way of forcing the final layer to only have positive weights


.data is deprecated, and the forum experts will threaten you with
the specter of computation-graph gremlins if you use it.
If you really want to do this, something like:
with torch.no_grad():
    self.classify.weight.copy_ (self.classify.weight.data.clamp(min=0))


might be better.


and therefore only looks for “positive” features to do classification ?


Is this the key to what you are trying to do?  What would it mean to
build a classifier that “only looks for ‘positive’ features?”


wouldn’t there be problems because sigmoid with only positive input only outputs +50% probabilities?


Well, if you only look for positive features, I suppose that it would
be natural to only find positive features, and therefore only output
probabilities greater than 50%.
But yes, this does seem problematic to me.


the bias should fix this problem, right?


I doubt anything I say where will be relevant to your actual problem,
but here are some observations:
Just because your last layer has only positive weights (as distinct
from biases) doesn’t mean that your output values (even if you had
no biases) would be positive.  Negative inputs to a positive-weight
layer will produce negative outputs.
But yes, a negative bias could flip the value of what would have been
a positive output to negative.
However, as it stands, I don’t see how your optimizer would know that
you want positive weights.  So even if your biases could “fix” the
problem, your optimizer could well just take steps that leave your
biases alone, and make your weights negative (even though you
then force them by hand to be positive).
Good luck.
K. Frank