hey @DLiebman i think the problem is that you’re passing a batch of 10 examples, but only a single label in the next_sentence_label argument. changing your code to the following works for me:
outputs = model(**encoding, next_sentence_label=torch.ones((10,1), dtype=torch.long), target_batch_size=10)