Hello,
I have been exploring the pipelines of transformers with a dataset of tweets.
After I charged the data and prepare it with train_test_split of scikit learn, I prepared everything as the code below:
# load the tokenizer
tokenizer = CamembertTokenizerFast.from_pretrained("camembert-base", do_lower_case=True)

# tokenize the dataset, truncate when passed `max_length = 512`, 
# and pad with 0's when less than `max_length`
train_encodings = tokenizer(train_texts, truncation=True, padding=True, max_length=512)
valid_encodings = tokenizer(valid_texts, truncation=True, padding=True, max_length=512)

#load the model and pass to CUDA after importing all the packages 
model = CamembertForSequenceClassification.from_pretrained(model_name, num_labels=len(label_dict)).to(device)

training_args = TrainingArguments(
    output_dir='./results',          # output directory
    num_train_epochs=4,              # total number of training epochs
    per_device_train_batch_size=10,  # batch size per device during training
    per_device_eval_batch_size=4,   # batch size for evaluation
    #warmup_steps=500,                # number of warmup steps for learning rate scheduler
    weight_decay=0.01,               # strength of weight decay
    logging_dir='./logs',            # directory for storing logs
    load_best_model_at_end=True,     # load the best model when finished training 
    logging_steps=100,               # log & save weights each logging_steps
    evaluation_strategy="steps",     # evaluate each `logging_steps`
    learning_rate=2e-5
)

trainer = Trainer(
    model="camembert-base",              # the instantiated Transformers model to be trained
    args=training_args,                  # training arguments, defined above
    train_dataset=train_dataset,         # training dataset
    eval_dataset=valid_dataset,          # evaluation dataset
    compute_metrics=compute_metrics,     # prepare the function to computes metrics of interest
)

At this point I have no error at all I check every tensor and every output? all go.
But when I come to the final step of training, the model keeps running with no end
# train the model
trainer.train()

and display this…
***** Running training *****
  Num examples = 61601
  Num Epochs = 4
  Instantaneous batch size per device = 10
  Total train batch size (w. parallel, distributed & accumulation) = 10
  Gradient Accumulation steps = 1
  Total optimization steps = 24644
Exception in thread Thread-15:
Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.7/dist-packages/torch_xla/distributed/parallel_loader.py", line 139, in _loader_worker
    _, data = next(data_iter)
  File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 521, in __next__
    data = self._next_data()
  File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 561, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "<ipython-input-15-37abd243a789>", line 7, in __getitem__
    item = {k: torch.tensor(v[idx]) for k, v in self.encodings.items()}
  File "<ipython-input-15-37abd243a789>", line 7, in <dictcomp>
    item = {k: torch.tensor(v[idx]) for k, v in self.encodings.items()}
IndexError: list index out of range

I couldn’t interpret the exception nor the error.
Can anyone help me through this, please?