That could be one reason.
Also, I’m a bit sceptical about these lines of code:
bs = torch.stack(bert_outputs).view(-1)
...
loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1))
outputs = (loss,) + outputs

Could you print the shape of bs, pooled_layer_output and logits?
Also, what is the last line of code doing?