Why are you getting just the last encoder states in the summarization code?
The line is here:

https://github.com/huggingface/transformers/blob/v2.3.0/examples/summarization/modeling_bertabs.py#L142

By changing the line to `encoder_hidden_states = encoder_output` I was able to fine-tune the model successfully, as well as, run the inference code from the `run_summarization.py` script.

So just wondering why you're indexing into the the encoder output rather than passing all of it along to the decoder?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
