I changed the classifier to :
self.bert = BertForSequenceClassification.from_pretrained('bert-base-uncased')
and for the forward function:
 x = self.bert(x, attention_mask=mask)
but I got  another problem which is that the output of self.bert is
a tuple containing a tensor and I want to extract the tensor from that tuple:
(tensor([[0.6515, 0.6653],
        [0.2553, 0.1504],
        [0.4994, 0.4797],
        [0.6337, 0.2966],
        [0.2209, 0.5274],
        [0.5424, 0.3485],
        [0.3630, 0.1887],
        [0.2545, 0.7262],
        [0.2021, 0.4187],
        [0.5003, 0.1696],
        [0.4143, 0.5789],
        [0.4960, 0.4392],
        [0.4121, 0.5264],
        [0.5776, 0.4939],
        [0.3785, 0.4943],
        [0.4546, 0.4430],
        [0.6860, 0.3314],
        [0.4527, 0.3885],
        [0.2221, 0.6505],
        [0.3507, 0.2968],
        [0.3292, 0.6263],
        [0.3475, 0.3593],
        [0.1828, 0.5570],
        [0.5490, 0.1212],
        [0.2501, 0.2750],
        [0.3481, 0.4373],
        [0.3475, 0.7435],
        [0.4671, 0.2719],
        [0.3621, 0.4031],
        [0.3909, 0.2389],
        [0.3114, 0.5124],
        [0.1240, 0.3718],
        [0.1745, 0.2922],
        [0.3455, 0.4789],
        [0.4206, 0.3542],
        [0.2758, 0.2613],
        [0.2718, 0.2825],
        [0.2359, 0.4479],
        [0.5177, 0.3300],
        [0.5761, 0.4069],
        [0.5057, 0.5138],
        [0.1883, 0.3227],
        [0.6071, 0.4967],
        [0.4363, 0.3138],
        [0.2681, 0.2961],
        [0.3116, 0.4401],
        [0.2646, 0.8440],
        [0.1167, 0.4557],
        [0.3569, 0.2142],
        [0.4936, 0.2005]], device='cuda:0'),)

and here is the error :
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-9-360b9852dfa7> in <module>()
    137 
    138 print('Running...')
--> 139 big_training_loop(555, optimizer, model, cross_entropy, train_loader, val_loader, accuracy_score=accuracy_score)
    140 

5 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in linear(input, weight, bias)
   1686         if any([type(t) is not Tensor for t in tens_ops]) and has_torch_function(tens_ops):
   1687             return handle_torch_function(linear, tens_ops, input, weight, bias=bias)
-> 1688     if input.dim() == 2 and bias is not None:
   1689         # fused op is marginally faster
   1690         ret = torch.addmm(bias, input, weight.t())

AttributeError: 'tuple' object has no attribute 'dim'

how should I fix it ?