Hi everyone,
I’m trying to fine-tune a T5 model. I followed most (all?) the tutorials, notebooks and code snippets from the Transformers library to understand what to do, but so far, I’m only getting errors. The end goal is giving T5 a task such as finding the max/min of a sequence of numbers, for example, but I’m starting with something really small, just to see if I understand how things work.
I’m using Transformers v4.2.2 (Tokenizers v0.9.4).
This is what I have understood so far and which I think is correct (excuse the French, I’m working on that too  ):
tokenizer = T5TokenizerFast.from_pretrained("t5-base")
model = T5ForConditionalGeneration.from_pretrained("t5-base")

prefix = "translate English to French:"
inputs = [f"{prefix} How are you?", f"{prefix} My name is Ben", f"{prefix} My cat is great"]
outputs = ["Comment ca va?", "Je m'appelle Ben", "Mon chat est genial"]
model_inputs = tokenizer(inputs, padding=True, truncation=True, return_tensors="pt")
with tokenizer.as_target_tokenizer():
    labels = tokenizer(outputs, padding=True, truncation=True, return_tensors="pt")
model_inputs["labels"] = labels["input_ids"]

class MyDataset(torch.utils.data.Dataset):
    def __init__(self, examples):
        self.examples = examples
    
    def __getitem__(self, idx):
        return self.examples[idx]
    
    def __len__(self):
        return len(self.examples)

train = MyDataset([model_inputs])

training_args = Seq2SeqTrainingArguments(
    output_dir="output",
    overwrite_output_dir=True,
    per_device_train_batch_size=2,
    num_train_epochs=3,
    run_name="T5 Experiment",
)

trainer = Seq2SeqTrainer(
    model=model,
    args=training_args,
    train_dataset=train,
    tokenizer=tokenizer
)

trainer.train()

The error I’m getting is ValueError: too many values to unpack (expected 2) which happens in the forward method in transformers/models/t5/modeling_t5.py, in the line 877 (880 on the master branch):
batch_size, seq_length = input_shape

Looking at a few lines before the error, I see input_shape is just input_ids.size(), and my model_inputs["input_ids"] is indeed a two-dimensional PyTorch tensor, so I don’t understand why the unpacking crashes.
I have no idea whether I’m doing something wrong in the model, in the tokenization, in the training, I really am lost.
Any help is greatly appreciated!