BERT multi heads attentions
Hello,
I would like to analysis the effect of specific heads' attention.
Is it possible to turn off some heads attentions in a particular layer?
if yes, can you please tell me how to do that or share any helpful document? 

Thank you in advance
I think the [BERTology](https://huggingface.co/transformers/bertology.html) section could help, especially the [run_bertology.py](https://github.com/huggingface/transformers/blob/master/examples/run_bertology.py) script can perform pruning and includes other useful functions :)<|||||>I am beginner to BERT, can you please tell me to turn off  the second  head in the ninth layer for example.
Here is my model config

config = BertConfig.from_pretrained("bert-base-uncased",output_attentions=True,output_hidden_states=True, num_labels=2)

model = BertForSequenceClassification.from_pretrained("bert-base-uncased", config= config)
model.cuda()<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
