I’m trying to probe a pretrained model of XLMForSequenceClassification. I want to freeze all layers but the last classifying layer. What layer is that for XLMForSequenceClassification? When I call .named_parameters(), the last layer seems to be:
sequence_summary.summary.weight
sequence_summary.summary.bias 

This is unlike using a pretrained BERTForSequenceClassification since the last layer there is explicitly specified as a classifier in its name. What is sequence_summary? Can I assume this is the classifying layer?
Even if I leave this layer unfrozen, I still seem to get the error:
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn