VisionEncoderDecoder Error during training
Thank you for VisionEncoderModel. It is really nice!!
I am currently trying to develop an Indonesian VisionEncoderModel. However, during the validation step, I am getting the following error. I am wondering whether this is an expected behavior or a bug of the code.
Thank you very much in advance.
 
transformers.version = '4.15.0'
 
 
fename='microsoft/beit-base-patch16-224'
tzname='cahya/roberta-base-indonesian-1.5G'
 
feature_extractor=ViTFeatureExtractor.from_pretrained(fename)
tokenizer = RobertaTokenizer.from_pretrained(tzname)
processor = TrOCRProcessor(feature_extractor=feature_extractor, tokenizer=tokenizer)
processor.save_pretrained(out_processor_dir)
model = VisionEncoderDecoderModel.from_encoder_decoder_pretrained(fename, tzname)

training_args = Seq2SeqTrainingArguments(
predict_with_generate=True,
num_train_epochs = 1,
learning_rate = 0.00002,
evaluation_strategy="steps",
lr_scheduler_type = "constant",
gradient_accumulation_steps=4,
per_device_train_batch_size=64,
per_device_eval_batch_size=64,
fp16=True,
output_dir=out_model_dir,
logging_steps=500,
save_steps=2000,
eval_steps=1,
dataloader_num_workers=16
)

trainer = Seq2SeqTrainer(
model=model,
tokenizer=processor.feature_extractor,
args=training_args,
compute_metrics=compute_metrics,
train_dataset=train_dataset,
eval_dataset=eval_dataset,
data_collator=default_data_collator,
)

trainer.train()

**Error message i got:**
Traceback (most recent call last):
File "src/finetune_ot_syndata.py", line 135, in
trainer.train()
File "/opt/conda/lib/python3.8/site-packages/transformers/trainer.py", line 1399, in train
self._maybe_log_save_evaluate(tr_loss, model, trial, epoch, ignore_keys_for_eval)
File "/opt/conda/lib/python3.8/site-packages/transformers/trainer.py", line 1521, in _maybe_log_save_evaluate
metrics = self.evaluate(ignore_keys=ignore_keys_for_eval)
File "/opt/conda/lib/python3.8/site-packages/transformers/trainer_seq2seq.py", line 70, in evaluate
return super().evaluate(eval_dataset, ignore_keys=ignore_keys, metric_key_prefix=metric_key_prefix)
File "/opt/conda/lib/python3.8/site-packages/transformers/trainer.py", line 2158, in evaluate
output = eval_loop(
File "/opt/conda/lib/python3.8/site-packages/transformers/trainer.py", line 2332, in evaluation_loop
loss, logits, labels = self.prediction_step(model, inputs, prediction_loss_only, ignore_keys=ignore_keys)
File "/opt/conda/lib/python3.8/site-packages/transformers/trainer_seq2seq.py", line 167, in prediction_step
generated_tokens = self.model.generate(
File "/opt/conda/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/opt/conda/lib/python3.8/site-packages/transformers/generation_utils.py", line 1024, in generate
model_kwargs = self._prepare_encoder_decoder_kwargs_for_generation(
File "/opt/conda/lib/python3.8/site-packages/transformers/generation_utils.py", line 486, in _prepare_encoder_decoder_kwargs_for_generation
model_kwargs["encoder_outputs"]: ModelOutput = encoder(*encoder_args, **encoder_kwargs)
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
TypeError: forward() got an unexpected keyword argument 'attention_mask'
0%| | 2/132200 [00:11<203:26:04, 5.54s/it]
Thanks for reporting, this has been fixed. See https://github.com/huggingface/transformers/issues/15648#issuecomment-1039452797.<|||||>Thank you very much. I am so lucky that it is already fixed. <|||||>@syoon9 hey can i get your contact to talk about this?