@valhalla @sshleifer
Hi, I’m new to the seq2seq model. And I want to fine-tune Bart/T5 for the summarization task. There are some documents related to the fine-tuning procedure.
Such as
  
      
      github.com
  
  
    

huggingface/transformers 24

master/examples/seq2seq

  🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0.

  
  
    
    
  
  


  
      
      ohmeow.github.io
  
  
    

modeling.seq2seq.summarization | blurr 16

This module contains custom models, custom splitters, etc... summarization tasks.


  
  
    
    
  
  


And also thanks for the distilbart version.
But my custom dataset is in Japanese. Directly fine-tuning might be impossible. Is it necessary to train a new bpe tokenizer with Japanese data? But I don’t know how to do it.
The second way is to use an existing Japanese tokenizer like bert-japanese, but could I just use it for Bart? How to modify it?
The third way is to use a multilingual model like MBart or MT5. I haven’t tested it. Could I just fine-tune them with the Japanese dataset ?
Please forgive me if this is a stupid question. Thanks in advance.
Thanks in advance.