I also tried to use trocr-small-handwritten encoder and get this error, when tried to predict image by model:
‘VisionEncoderDecoderModel’ object has no attribute ‘enc_to_dec_proj’
from transformers import RobertaForCausalLM, VisionEncoderDecoderModel

model = VisionEncoderDecoderModel.from_pretrained(f'microsoft/trocr-small-handwritten')
model.decoder = RobertaForCausalLM.from_pretrained('xlm-roberta-base', is_decoder=True, add_cross_attention=True)


image1837×738 133 KB