You could try to normalize your dataset, which should help the training.
If that doesn’t help, try to scale down the problem and overfit a small dataset of e.g. just 10 samples.
Once your model is able to learn these samples perfectly, you could try to scale up the use case again.
Also, I would generally recommend to remove the sigmoid activation and use nn.BCEWithLogitsLoss instead of nn.BCELoss, as this would yield additional numerical stability.