Thanks for the response!
So I think I figured it out, and it seems like it was a silly issue. I am very new to PyTorch and I have been jumping into new transformer models while also playing with “old” CNNs. The transformer BERT model, which I had an extensive guide for, worked fine and outputted logits that looked like the following for a 2 label classification task.
tensor([[ 0.5015, -0.0165],
        [ 0.5486,  0.0320],
        [ 0.4227,  0.1604],
        [ 0.2781, -0.0317],
        [ 0.2667,  0.2109],
        [ 0.1847, -0.1724],
        [ 0.2727, -0.0598],
        [ 0.3827,  0.1195],
        [ 0.2796, -0.2183],
        [ 0.6082, -0.1816],
        [ 0.4710, -0.0551],
        [ 0.1589,  0.0477],

For my CNN, I mistakenly made my output to be 2 (to make the output look like the above working transformer), instead of 1; which I thought was just a difference in PyTorch from Keras, as the last layer in Keras for a 2 label classification problem looks like keras.layers.Dense(1, activation="sigmoid").
With my number of classes configuration set to 1, instead of 2 now, BCEWithLogitsLoss seems to work as intended via:
criterion = nn.BCEWithLogitsLoss()
        # `batch` contains two pytorch tensors:
        #   [0]: input ids
        #   [1]: labels
        b_input_ids = batch[0].cuda()
        b_labels = batch[1].cuda().type(torch.cuda.FloatTensor)

        # clear previously calculated gradients
        model.zero_grad()

        # forward propagation (evaluate model on training batch)
        logits = model(b_input_ids)

        # calculate cross entropy loss
        loss = criterion(logits, b_labels)

While this is not the place for huggingface/transformers discussion, I am curious as to why, for binary classification, the model outputs 2 columns of logits that look like the above instead of 1 column. I also realize that the way the transformers package has setup its API (https://huggingface.co/transformers/model_doc/bert.html 1) makes it so that if:
num_labels = 1; its a regression
num_labels >= 2; its classification
and thus may be why.
Thanks for your time!