Sorry, I am not sure about those specific models and I do not have the time to go into it in detail. I can find that the fairseq datasets for MT by default include a shifted version of the golden targets for teacher forcing.

  

      github.com
  

  
    pytorch/fairseq/blob/1bba712622b8ae4efb3eb793a8a40da386fe11d0/fairseq/data/language_pair_dataset.py#L182-L183 2



      
      
          input_feeding (bool, optional): create a shifted version of the targets
              to be passed into the model for teacher forcing (default: True).
      
    
  


  

  
    
    
  

  


I am not sure about the implementation/training/finetuning of all these models.
But, considering that one of the advantages of teacher forcing is faster convergence, fine-tuning (on limited data) with teacher forcing seems a good idea.