Hello everyone,
I am working on multiclass text classification, currently using XLM-Roberta as a classifier. I have a doubt concerning padding strategies.
My first intuition was to tokenize my training and validation sets separately (as they were two distinct super-batches) using padding = True; the result of this was having all training examples padded to length l1 (the length of the longest sequence in the training set), and validation examples padded to a different l2.
An alternative approach (and the one that seems to be used in the GlueDataset and related methods) is to use padding = max_length, and thus have all examples padded to the same provided length (possibly, 512, which is the maximum sequence length allowed for this model).
Would you mind sharing your thoughts on what strategy might work best and makes more sense from a “theoretical” point of view?
Thank you very much!