Bart Caching: do we need encoder outputs after step 1?
since projected cross attention k,v are cached, this doesn't seem like it's actually needed.
Maybe we could save some mem if we deleted them after we were done.
Very good point! But I'm actually not sure if the memory consumption of `num_layers * cached k, v states` already makes memory consumption of `encoder_outputs` negligible. Would be cool to try out.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
