I am using BartForConditionalGeneration for text summarization.
The ‘use_cache’ option is True by default when pre-training the Bart.
Purpose: The purpose of ‘use_cache’ option seems speed-up decoding.
(https://github.com/huggingface/transformers/blob/d822ab636b6a14ed50f7bca0797c1de42c19de61/src/transformers/modeling_bart.py#L120-L122 30)
Implementation: When ‘use_cache’ = True, the decoder use only the last time steps of input_ids & positional embedding.
(https://github.com/huggingface/transformers/blob/d822ab636b6a14ed50f7bca0797c1de42c19de61/src/transformers/modeling_bart.py#L551-L553 17)
Can anyone explain how above purpose & implementation are related?
Thank you