Solved by ptrblck in post #2 
              
              
                Are you using the sigmoid for both layers? 
Note that nn.BCEWithLogitsLoss does not expect probabilities, but raw logits. 
Could you post the training code, so that we can have a look?