Regarding attention received by the distilbert model
I receive the attention of all the 6 layers with all the 12 heads while exporting the tfdistilbert model. I just want to take the attention of dimension equal to the sequence length. Which layer and head attention is the most effective attention value that I should take in order to get the best value of attention scores with respect to my sentence.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
