unexpected keyword argument 'encoder_hidden_states' when using PreTrainedEncoderDecoder
## ❓ Questions & Help

After I have defined my seq2seq class using Encoder Decoder Architecture in the following way:
```
from transformers import PreTrainedEncoderDecoder
model = PreTrainedEncoderDecoder.from_pretrained('bert-base-uncased','gpt2')
```
I try to forward tensor through the model in this following way:
```
model(question_batch, answer_batch)
```
 but I got this error:

```
---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-16-bb34e9576e8b> in <module>()
----> 1 model(test_history, test_knowledge)

2 frames

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    539             result = self._slow_forward(*input, **kwargs)
    540         else:
--> 541             result = self.forward(*input, **kwargs)
    542         for hook in self._forward_hooks.values():
    543             hook_result = hook(self, input, result)

TypeError: forward() got an unexpected keyword argument 'encoder_hidden_states'
```

can anyone help me?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>I am facing the same issue. Probably `PreTrainedEncoderDecoder` supports only `Bert` to `Bert` models.<|||||>I'm getting this error too
```
Converting vae...
Converting text encoder...
Downloading100% 1.71G/1.71G [00:24<00:00, 69.1MB/s]
Downloading100% 4.55k/4.55k [00:00<00:00, 746kB/s]
Downloading100% 1.22G/1.22G [00:18<00:00, 66.3MB/s]
Downloading100% 342/342 [00:00<00:00, 57.0kB/s]
Saving diffusion model...
Restored system models.
Checkpoint successfully extracted to /content/gdrive/MyDrive/sd/stable-diffusion-webui/models/dreambooth/TTFM/working
Returning ['default', True, True, 1, '', '', 0, 0, 1, True, True, 50, False, False, 5e-06, 1e-06, 0.0001, '', 5e-05, 1.0, 1.0, 1, 0.5, 1.0, 0.5, 'constant_with_warmup', 0, 75, 'fp16', 100, True, '', 1.0, 512, 1, '', 420420, True, False, True, 25, True, False, True, 5, False, False, False, False, 1, False, 1.0, True, False, True, False, False, '', 7.5, 60, '', '', '', '', '', '', 1, 0, 0, -1, 7.5, 60, '', '', '', '', 7.5, 60, '', '', '', '', '', '', 1, 0, 0, -1, 7.5, 60, '', '', '', '', 7.5, 60, '', '', '', '', '', '', 1, 0, 0, -1, 7.5, 60, '', '', '', 'Loaded config.']
Saved settings.
Custom model name is TTFM
Starting Dreambooth training...
Initializing dreambooth training...
Replace CrossAttention.forward to use default
Instance Bucket 0: Resolution (512, 512), Count: 723
Target Bucket 0: Resolution (512, 512), Count: 0
We need a total of 0 images.
Nothing to generate.
Exception importing 8bit adam: No module named 'bitsandbytes'
WARNING:extensions.sd_dreambooth_extension.dreambooth.train_dreambooth:Exception importing 8bit adam: No module named 'bitsandbytes'
Traceback (most recent call last):
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/train_dreambooth.py", line 304, in inner_loop
    import bitsandbytes as bnb
ModuleNotFoundError: No module named 'bitsandbytes'
Preparing dataset
Preparing dataset
Preparing Dataset (With Caching)
100% 723/723 [00:46<00:00, 15.47it/s]
Train Bucket 1: Resolution (512, 512), Count: 723
Total images: 241
Total dataset length (steps): 241
Sched breakpoint is 108450
  ***** Running training *****
  Instance Images: 723
  Class Images: 0
  Total Examples: 723
  Num batches each epoch = 241
  Num Epochs = 300
  Batch Size Per Device = 3
  Gradient Accumulation steps = 3
  Total train batch size (w. parallel, distributed & accumulation) = 9
  Text Encoder Epochs: 210
  Total optimization steps = 216900
  Total training steps = 216900
  Resuming from checkpoint: False
  First resume epoch: 0
  First resume step: 0
  Lora: False, Adam: False, Prec: bf16
  Gradient Checkpointing: True
  EMA: True
  LR: 4.5e-05)
Steps:   0% 0/216900 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/scripts/dreambooth.py", line 561, in start_training
    result = main(config, use_txt2img=use_txt2img)
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/train_dreambooth.py", line 973, in main
    return inner_loop()
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/memory.py", line 116, in decorator
    return function(batch_size, grad_size, prof, *args, **kwargs)
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/train_dreambooth.py", line 829, in inner_loop
    noise_pred = unet(noisy_latents, timesteps, encoder_hidden_states).sample
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/accelerate/utils/operations.py", line 490, in __call__
    return convert_to_fp32(self.model_forward(*args, **kwargs))
  File "/usr/local/lib/python3.8/dist-packages/torch/amp/autocast_mode.py", line 14, in decorate_autocast
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/diffusers/models/unet_2d_condition.py", line 481, in forward
    sample, res_samples = downsample_block(
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/diffusers/models/unet_2d_blocks.py", line 781, in forward
    hidden_states = torch.utils.checkpoint.checkpoint(
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/checkpoint.py", line 249, in checkpoint
    return CheckpointFunction.apply(function, preserve, *args)
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/checkpoint.py", line 107, in forward
    outputs = run_function(*args)
  File "/usr/local/lib/python3.8/dist-packages/diffusers/models/unet_2d_blocks.py", line 774, in custom_forward
    return module(*inputs, return_dict=return_dict)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/diffusers/models/transformer_2d.py", line 265, in forward
    hidden_states = block(
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/diffusers/models/attention.py", line 285, in forward
    attn_output = self.attn1(
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
TypeError: forward_default() got an unexpected keyword argument 'encoder_hidden_states'
Steps:   0% 0/216900 [00:01<?, ?it/s]
Training completed, reloading SD Model.
Restored system models.
Returning result: Exception training model: 'forward_default() got an unexpected keyword argument 'encoder_hidden_states''.
```