pregenRobot:

Where is the number 442 even coming from? I have clearly stated that I want each batch to have a size of 13 rows.


I think it’s the flattened target tensor: [13*43=442].
Based on the docs for BertForSequenceClassification, the targets are expected as [batch_size] so they are most likely internally flattened.
I’m not familiar with the implementation, but since it’s a classification use case I guess the targets have the same (or similar) requirements as the targets for nn.CrossEntropyLoss.
I.e. they are expected in the shape [batch_size] as LongTensors containing the class indices in the range [0, nb_classes-1] for a multi-class classification use case.