Yeah I have been running everything with --freeze_embeds.
I’m also trying to figure out how to trim the embeddings, as most of them aren’t used, but blocked on

  
      github.com/google/sentencepiece
  
  
    
  
	  
  

  
    
      How to create new model file with restricted vocabulary? 6
    

    
      
        opened 
        
          
        
        Jul 23, 2020
      
      


      
        
          
          sshleifer
        
      
    
  



  Similar to #474, I want to restrict my vocabulary, and then save a new model file that uses the restricted vocabulary.
I...





  
  
    
    
  
  


I’m also happy to finetune mbart-cc25 on a public dataset for you, if that would help.
Also, have you tried using Marian?