cpeters:

If I remove the sigmoidal activation and re-apply it to the model output (i.e. output = model(images) becomes output = torch.sigmoid(model(images)), the calculated loss after 20 epochs of training remains the same but there’s a drastic difference in the output images:


If I understand the workflow correctly, you are now feeding the raw logits to nn.BCEWithLogitsLoss, which yields the same loss as in the float32 training, and use the output = torch.sigmoid(model(images)) tensor only to visualize the samples?
If so, do you see any difference by converting the output to .float() before feeding it into the torch.sigmoid operation?