Sure, sorry! I thought it was not a code specific question but rather about the parameters of the Trainer class 
Here is the code I use:
from transformers import Trainer, TrainingArguments
training_args = TrainingArguments(
output_dir=’./out’,
num_train_epochs=1000,
per_device_train_batch_size=2,
per_device_eval_batch_size=2,
warmup_steps=500,
weight_decay=0.01,
logging_dir=’./logs’,
logging_steps=10,
)
trainer = Trainer(
model=model,
args=training_args,
train_dataset=training_data,
eval_dataset=validation_data)
trainer.train()
I would expect that there are 1000 epoch that goes through the full train_dataset but the output goes very fast and prints “epoch 1/1000, epoch 2/1000, epoch 3/1000, …” which give the impression that the epoch is simply a training step rather than an actual epoch. However, having 20k training instances I expect to see 20k steps for each epoch and this normally take some time.