I think you are spot on.
I did not apply sigmoid when doing model(x) during evaluation. So that must be the issue since indeed the activation is carried by the loss.
I understand BCEWithLogits is better for stability but then indeed one has to he careful when evaluating