Hi @thomasdaryl,
I’ve seen a paper do what you’re looking for, as an extension of the DRG model they use the attention weights of the model in order to identify which tokens are contributing to the classification results most.
The paper is seen here:
  
      arxiv.org
  
  
    
1908.09368.pdf 8

415.77 KB

  
  
    
    
  
  


They also have a helpful git repo here: https://github.com/agaralabs/transformer-drg-style-transfer 17
Which specifies that they used BERT to do exactly the task that you’re looking for as I understand it:

We have used BERT 2 for classification. This classification trainings helps to find the attributes from the sentence. We choose one particular head of BERT model, for which the tokens which have high attention weights are those that are stylistic attributes of the sentence.

Hope this helps