My implementation of NCE loss did not use a stable log-sum-exp computation, which may have caused your nan problem. I changed the implementation to use torch.nn.functional.binary_cross_entropy_with_logits(), which should be numerically stable.