Hello,
I would like to train bart from scratch.
It seems the official example script is not available yet (if any, please tell me!).
So I try to have one by modifying the example scripts run_mlm.py and run_clm.py.
And not sure how to set the data collator part for bart.
In run_mlm.py , DataCollatorForLanguageModeling is used:

  
      github.com
  
  
    huggingface/transformers/blob/0c9bae09340dd8c6fdf6aa2ea5637e956efe0f7c/examples/language-modeling/run_mlm.py#L342 17

    # https://huggingface.co/docs/datasets/package_reference/main_classes.html#datasets.Dataset.map    tokenized_datasets = tokenized_datasets.map(        group_texts,        batched=True,        num_proc=data_args.preprocessing_num_workers,        load_from_cache_file=not data_args.overwrite_cache,    )# Data collator# This one will take care of randomly masking the tokens.data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm_probability=data_args.mlm_probability)# Initialize our Trainertrainer = Trainer(    model=model,    args=training_args,    train_dataset=tokenized_datasets["train"] if training_args.do_train else None,    eval_dataset=tokenized_datasets["validation"] if training_args.do_eval else None,    tokenizer=tokenizer,    data_collator=data_collator,)



  
  
    
    
  
  


In run_clm.py , default_data_collator is used:

  
      github.com
  
  
    huggingface/transformers/blob/0c9bae09340dd8c6fdf6aa2ea5637e956efe0f7c/examples/language-modeling/run_clm.py#L310 7

)# Initialize our Trainertrainer = Trainer(    model=model,    args=training_args,    train_dataset=lm_datasets["train"] if training_args.do_train else None,    eval_dataset=lm_datasets["validation"] if training_args.do_eval else None,    tokenizer=tokenizer,    # Data collator will default to DataCollatorWithPadding, so we change it.    data_collator=default_data_collator,)# Trainingif training_args.do_train:    trainer.train(        model_path=model_args.model_name_or_path if os.path.isdir(model_args.model_name_or_path) else None    )    trainer.save_model()  # Saves the tokenizer too for easy upload# Evaluation



  
  
    
    
  
  


Can someone give some advice about how to set the data collator for bart?
Thanks.