How to obtain attention values for each layer 
Hi all, 

Please correct me if I am wrong. 

From my understanding, The encoded values for each layer (12 of them for base model) would be returned when we run our results through the pre-trained model. 
However, I would like to examine the self-attention values for each layer. Is there a way I can extract that out? 

Regards
Not really.
You should build a new sub-class of `BertPreTrainedModel` which is identical to `BertModel`but send back self-attention values in addition to the hidden states.
<|||||>I see. Thank you! <|||||>Hi, 

Just to add on. If this is what I would be doing, would it be advisable to fine-tune the weights for the pretrained model? 

Regards<|||||>Probably.
It depends on what's your final use-case.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
