Solved by ptrblck in post #2 
              
              
                It’s usually better to remove the sigmoid and use nn.BCEWithLogitsLoss or F.binary_cross_entropy_with_logits. 
However, your model might be too small, so I would recommend to add a relu and another linear layer to it. 
Let me know, if that helps.