- past becomes past_key_values in all GPT-2 models.
Regarding the tokenizer classes:
- The tokenizer attribute max_len becomes model_max_length.
- The tokenizer attribute return_lengths becomes return_length.
- The tokenizer encoding argument is_pretokenized becomes is_split_into_words.
Regarding the Trainer class:
- The Trainer argument tb_writer is removed in favor of the callback TensorBoardCallback(tb_writer=...).
- The Trainer argument prediction_loss_only is removed in favor of the class argument args.prediction_loss_only.
- The Trainer attribute data_collator should be a callable.
- The Trainer method _log is deprecated in favor of log.
- The Trainer method _training_step is deprecated in favor of training_step.
- The Trainer method _prediction_loop is deprecated in favor of prediction_loop.
