You can print the model and you will see the structure:
tokenizer = XLMTokenizer.from_pretrained(‘xlm-mlm-100-1280’)
model = XLMForSequenceClassification.from_pretrained(‘xlm-mlm-100-1280’)
print(model)
…
…
…
(sequence_summary): SequenceSummary(
(summary): Linear(in_features=1280, out_features=2, bias=True)
(activation): Identity()
(first_dropout): Dropout(p=0.1, inplace=False)
(last_dropout): Identity()
)
You can still see we have the linear layer with in_features of 1280 and out_features as 2.
When using ‘bert-nased-cased’ or ‘xlnet-base-cased’, we see the last layer has different names (classifier and logits_prog respectively). Perhaps these names are as from the paper that first spoke about these models…So, when printing the named_parameters, you could also print the model, to see how the last layer is structured and its name - then, use them accordingly.
On your Runtime Error, could you share your code so we can see where the issue occurs?