You can look at the attention weights. This is not exactly “explanation”, but also not entirely not explanation either  That is, it can give you some notion of how “important” a word is, but is very limited in its explanatory power. Models like these are difficult to interpret.
Pass output_attentions=True to get the attention patterns back when you call the model. You’ll probably want the attention between each word and the EOS token since this embedding is the sequence representation that BartForSequenceClassification uses. But which layers and heads are best to use, or how to aggregate their patterns, goes beyond my knowledge.
As an aside for any interested readers, we have an awesome tool for visualizing transformers 1 which is really helpful to build intuition about attention patters in transformers.