Is there any way that I can extract the hidden output from the self-attention layer?
Hello,

From my understanding, for the ```GPT2LMHeadModel```, the output ```past``` allows me to retrieve the key and value vectors that are used in the self-attention block (which is prior to the feedforward block). 

Is there any way I can extract the output of the self-attention block **at a particular head of a single layer** of ```GPT2LMHeadModel``` (if I am understanding this correctly, the output ```hidden_states``` only returns the output after the input had gone into the feedforward block... but what I want is to extract the output from the self-attention block, which happens before the feedforward block).

Thank you,
That would be the attentions, which you can output by specifying `output_attentions=True` in your configuration object.<|||||>Hello,

Thank you very much for your reply.

What I want to obtain though, is not the individual attention weights themselves but rather the final product of the self-attention layer at each head (the transformed embeddings that the self-attention layer produces, before they go into the feedforward layer for final processing).

Is there any way that I can get this final product of the self-attention layer at each head?

Thank you,<|||||>You mean you want to obtain the result after the softmax multiplied by the value vector?<|||||>Hello,

I would like to obtain the result that is obtained after the sum of (value) * (softmax) got multiplied by the matrix H (i.e. the final output embedding of the self-attention layer from a single head)

Thank you,<|||||>Then that is literally the attentions I mentioned earlier, see in the [source code](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_gpt2.py#L163-L164).<|||||>Hello,

so the output of a single attention-head, which is the thing I want to extract, can be formulated as the following:

O = AE(W^V)H

where

O = output of a single attention-head
A = matrix that stores attention weights for all tokens in sequence
E = matrix that stores the embeddings of all tokens in a sequence
W^V = matrix that we multiply with E to generate the value vector of all tokens of the sequence
H = projection matrix that is used to generate the final product of a single attention-head

If I am not mistaken, ```attention``` gives out the matrix A...
but what I am looking to get is the output O.....

Is there anyway that I can get the output O? or does ```attention``` give out the output O, like you described before?

Thank you and sorry for the long question, your help is much appreciated.

<|||||>To be even more clear, I just want the output of each head within the layers of transformer. Is there any way that I can get the output of each individual head?

Thank you,<|||||>Hello, if I can't get the output of individual attention-head explicitly, is there any way that I can retrieve the matrix H, where H is from the formula below: 

O = AE(W^V)H

O = output of a single attention-head
A = matrix that stores attention weights for all tokens in sequence
E = matrix that stores the embeddings of all tokens in a sequence
W^V = matrix that we multiply with E to generate the value vector of all tokens of the sequence
H = projection matrix that is used to generate the final product of a single attention-head

Thank you,<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>@h56cho 

Hello

I also want to know if I can get such hidden outputs
Do you have any progress with it?

Thank you in advance