If you are using F.cross_entropy, you should pass the model outputs as logits to this criterion. Currently it seems you are applying F.sigmoid on them.
Could you remove it and try your training again?