For bidirectional=False layers, the tensors should be equal, yes.

nn.BCEWithLogitsLoss expects raw logits as the model output, so you should not apply a sigmoid on the output before passing it to the loss function.
During the prediction phase you could apply a sigmoid and use a threshold to get the class labels, e.g.:

threshold = 0.5
pred = torch.sigmoid(output) > threshold

Your rounding approach would also work, but the threshold would allow you to pick a point on the ROC curve.