DataCollatorForSOP marked as deprecated but DataCollatorForLanguageModeling does not offer the same functionality
The `DataCollatorForSOP` is marked as deprecated and it is recommended to use the `DataCollatorForLanguageModeling` instead. [Link to the data_collator.py](https://github.com/huggingface/transformers/blob/4bae96ec2bee265f938fc262201538819419089a/src/transformers/data/data_collator.py)

As far as I can tell, the labels for the sentence order prediction task (`sentence_order_label`) are not set in `DataCollatorForLanguageModeling`. 

Will this be added to `DataCollatorForLanguageModeling` in a future release, or what is the correct procedure when a data collator is needed for both the mlm and sop tasks at the same time, as in ALBERT training?


The `sentence_order_label` will be left as is and collated if your dataset provides them. This is tested [here](https://github.com/huggingface/transformers/blob/2550b41aa2ec34f05ddfd3ec5875ddb32ad78d58/tests/test_data_collator.py#L268) which is adapted from the old test of `DataCollatorForSOP`.<|||||>Oh, now I see it too. I was kind of under the assumption that the DataCollator could also be used on non-preprocessed data to tokenize and preprocess batches on demand.