As a work around, I introduced the training boolean parameter in the create_dataset function. For training I do not pad and let the trainer handle it. For evaluation I pass false and I pad them by the tokenizer. If you set the argument to False for eval data set as well, it crashes