from transformers import AutoModelForMaskedLM
self.xlm_roberta = AutoModelForMaskedLM.from_pretrained(“xlm-roberta-base”)

When I changed the model like this, I got an error.
I don’t really know what I can do to improve it.


IndexError                                Traceback (most recent call last)
 in 
4     print(f’FOLD: {fold}’)
5     print(’-’*50)
----> 6     run(train, fold)
 in run(data, fold)
18         result_dict = trainer.train(
19             args, train_dataloader,
—> 20             epoch, result_dict
21         )
22         torch.cuda.synchronize()
 in train(self, args, train_dataloader, epoch, result_dict)
32             outputs_start, outputs_end = self.model(
33                 input_ids=input_ids,
—> 34                 attention_mask=attention_mask,
35             )
36
/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
725             result = self._slow_forward(*input, **kwargs)
726         else:
 → 727             result = self.forward(*input, **kwargs)
728         for hook in itertools.chain(
729                 _global_forward_hooks.values(),
 in forward(self, input_ids, attention_mask)
32
33         sequence_output = outputs[0]
—> 34         pooled_output = outputs[1]
35
36         # sequence_output = self.dropout(sequence_output)
/opt/conda/lib/python3.7/site-packages/transformers/file_utils.py in getitem(self, k)
1738             return inner_dict[k]
1739         else:
 → 1740             return self.to_tuple()[k]
1741
1742     def setattr(self, name, value):
IndexError: tuple index out of range


Model code

  
      

      kaggle.com
  

  
    

chaii QA - 5 Fold XLMRoberta Torch | Infer

  Explore and run machine learning code with Kaggle Notebooks | Using data from xlm roberta squad2


  

  
    
    
  

  



Model

  

      huggingface.co
  

  
    

xlm-roberta-base · Hugging Face