What does the variable 'present' represent?
Hello,

does the variable 'present' shown in [this](https://github.com/huggingface/transformers/blob/4e69104a1fba717026d6909d06288788e684c749/src/transformers/modeling_gpt2.py#L187) line of Hugging Face GPT-2 code represent final output of a single attention-head? (i.e. **not** the final output of the _output head_ , but the final output of the individual _attention head_, which is placed right before the feedforward block of the same layer).

If yes, is there anyway that I can retrieve the value of the variable 'present'?
Would it be possible that Hugging Face will make the value available for everyone?

Thank you,