Hello KFrank,
thank you for your elaborate response. I have uploaded an instance of tensors for which the two losses diverge here 1. This is the resulting output for it:
import torch

x_view  = torch.load("x_view.pt")
x_recon = torch.load("x_recon.pt")

BCE1 = torch.nn.functional.binary_cross_entropy_with_logits(x_recon, x_view, reduction='sum')
BCE2 = torch.nn.functional.binary_cross_entropy(torch.sigmoid(x_recon), x_view, reduction='sum')

print("BCE loss with logits: ", BCE1)  # -2.3662e+08
print("BCE loss with sigmoid: ", BCE2) # -379848.2500
print("Loss difference: ", BCE1-BCE2)  # -2.3624e+08

The x_recon values are all very small (around -1.0e+05). However, I am not sure why the version without logits behaves so differently in this regime. Do you have any insights in this regard?
In the mean time I’ll try to figure out the minimal changes to the official example that are necessary to reproduce the odd behaviour and post the code here.