@nielsr
yes, I’ve just looked at your changes  However, I still have the same problems with model.generate () in relation to the validation step and the possibility of exporting the model to ONNX 

  
    
    
    VisionEncoderDecoder/TrOCR Models
  
  
    Hi, 
@nielsr 
I try currently to setup your VisionEncoderDecoderModel with a pretrained bert model as decoder but have some struggle with the model.generate (greedy search) part. 
I have setup all i have in a Colab Notebook (else it would be to much for posting directly) 
Colab Notebook  (Note: it is open if anyone is interested in collaborating) 
Questions: 
What are the difference between decoder_input_ids and labels ? (I think decoder_input_ids are for training the tokenized labels and labels…
  


PS: i have removed the decoder_input_ids after your push today  but this notebook is not up to date