Hi,
that’s not really possible, unless the model has a language modeling head that has been trained. If that’s not the case, it will load a randomly initialized language modeling head, which gives random predictions.
The "deepset/bert-base-cased-squad2" checkpoint has a fine-tuned question-answering head, but not a trained language modeling head.