Solved by ptrblck in post #4 
              
              
                Which value are you using for the label smoothing and which criterion are you using? 
Are you using a sigmoid at the end of your model and nn.BCELoss? 
If so, could you remove the sigmoid and use nn.BCEWithLogitsLoss? 
The pred calculation looks a bit weird. If you are already using a sigmoid output…