Error when calculating loss and running backward
I'm using the sentence classification example. I used my own dataset for emotionclassification (4 classes).

The hyper-parameters are as follows:
<pre>
args.max_seq_length = 100
args.do_train = True
args.do_eval = True
args.do_lower_case = True
args.train_batch_size = 32
args.eval_batch_size = 8
args.learning_rate = 2e-5
args.num_train_epochs = 3
args.warmup_proportion = 0.1
args.no_cuda = False
args.local_rank = -1
args.gpu_id = 1
args.seed = 412
args.gradient_accumulation_steps = 1
args.optimize_on_cpu = False
args.fp16 = False
args.loss_scale = 128
</pre>

I prepared my dataset accordingly and properly:
<pre>
12/04/2018 21:23:02 - INFO - __main__ -   *** Example ***
12/04/2018 21:23:02 - INFO - __main__ -   guid: train-1
12/04/2018 21:23:02 - INFO - __main__ -   tokens: [CLS] but i don ' t [ sep ] u just did [ sep ] i don ##t want to talk to u [SEP]
12/04/2018 21:23:02 - INFO - __main__ -   input_ids: 101 2021 1045 2123 1005 1056 1031 19802 1033 1057 2074 2106 1031 19802 1033 1045 2123 2102 2215 2000 2831 2000 1057 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12/04/2018 21:23:02 - INFO - __main__ -   input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12/04/2018 21:23:02 - INFO - __main__ -   segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12/04/2018 21:23:02 - INFO - __main__ -   label: angry (id = 3)
</pre>

When I run the following code, a runtime error occurred:

<pre>
for step, batch in enumerate(tqdm(train_dataloader, desc="Iteration")):
&nbsp &nbsp batch = tuple(t.to(device) for t in batch)
&nbsp &nbsp input_ids, input_mask, segment_ids, label_ids = batch
&nbsp &nbsp loss = model(input_ids, segment_ids, input_mask, label_ids)
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-20-1977b86302ed> in <module>()
     17             try:
---> 18                 loss.backward()
     19             except RuntimeError:

/raid5/peixiang/anaconda3/lib/python3.6/site-packages/torch/tensor.py in backward(self, gradient, retain_graph, create_graph)
     92         """
---> 93         torch.autograd.backward(self, gradient, retain_graph, create_graph)
     94 

/raid5/peixiang/anaconda3/lib/python3.6/site-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)
     89         tensors, grad_tensors, retain_graph, create_graph,
---> 90         allow_unreachable=True)  # allow_unreachable flag
     91 

RuntimeError: cublas runtime error : the GPU program failed to execute at /opt/conda/conda-bld/pytorch_1532581333611/work/aten/src/THC/THCBlas.cu:411
</pre>

What might be the cause? The dataset? I run the MRPC example without any issue.
I probably know the bug. The final output layer is for binary classification but I use it for 4-class classification. I thought BERT can automatically decide between sigmoid and soft max. I will replace it with my own classifier tomorrow and see how it goes.<|||||>The mismatched output size between BERT and our dataset is the bug. Also, remember to set the num_labels to your output size:

<pre>
output_size = 4
model.classifier = nn.Linear(768, output_size)
model.num_labels = output_size 
</pre>