Hi,
In Google Colab notebook, I install (!pip transformers) and import XLNetForSequenceClassification model. When I instantiate the model the firs time (before training), I get the below:
Some weights of the model checkpoint at xlnet-base-cased were not used when initializing XLNetForSequenceClassification: [‘lm_loss.weight’, ‘lm_loss.bias’]

This IS expected if you are initializing XLNetForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
This IS NOT expected if you are initializing XLNetForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of XLNetForSequenceClassification were not initialized from the model checkpoint at xlnet-base-cased and are newly initialized: [‘sequence_summary.summary.weight’, ‘sequence_summary.summary.bias’, ‘logits_proj.weight’, ‘logits_proj.bias’]
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

After training, I save the model’s state_dict using torch.save(). When I load this model for inference (using torch.load), I get the same messages as above.
Why would I get the messages post training?