choke:

Also, my end goal is to train the reformer and then use it to generate embeddings rather than classification. Is this the correct approach to do so?


sorry i only just saw this - if your goal is to extract embeddings, then my suggestion would be to train / fine-tune the reformer as a language model on your corpus (no labels needed!). you can find tutorials on how to do this for both cases (from scratch vs fine-tune) here: 🤗 Transformers Notebooks — transformers 4.7.0 documentation 6
regarding your error, i now see you used DataCollatorForTokenClassification which is not the correct choice for sequence classification tasks. for these tasks, you can just use the default DataCollatorWithPadding which can be activated by passing the tokenizer to the Trainer:
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=dataset,
    tokenizer=tokenizer
)