DeBERTa from official example return random logits that change every time I run the same example.
EDIT: This happens each time the model  is loaded again from disk. Once model is loaded the logits  are the same each run for the same input but when I reload the model the logits change.

I tried to run the official example of Deberta and found that the output logins are just random numbers that change every time i run the model? Is this a bug?

Transformers version is: 4.9.2 also in 4.15.0 is the same and i tried the current latest version and it is the same.

     from transformers import DebertaTokenizer, DebertaForMaskedLM
    import torch
    tokenizer = DebertaTokenizer.from_pretrained("microsoft/deberta-base")
    model = DebertaForMaskedLM.from_pretrained("microsoft/deberta-base")
    model.eval()


    inputs = tokenizer("The capital of France is [MASK].", return_tensors="pt")
    labels = tokenizer("The capital of France is Paris.", return_tensors="pt")["input_ids"]

    with torch.no_grad():
        outputs = model(**inputs, labels=labels)

    loss = outputs.loss

    logits = outputs.logits

    print("logits",logits)`


example output:
1) tensor([[[-0.3574, -0.4279
2) tensor([[[-1.0341,  0.5181

I also get this as warning:
Some weights of the model checkpoint at microsoft/deberta-base were not used when initializing DebertaForMaskedLM: ['deberta.embeddings.position_embeddings.weight', 'lm_predictions.lm_head.LayerNorm.weight', 'lm_predictions.lm_head.LayerNorm.bias', 'lm_predictions.lm_head.bias', 'lm_predictions.lm_head.dense.weight', 'lm_predictions.lm_head.dense.bias']
- This IS expected if you are initializing DebertaForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing DebertaForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of DebertaForMaskedLM were not initialized from the model checkpoint at microsoft/deberta-base and are newly initialized: ['cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.weight', 'cls.predictions.bias', 'cls.predictions.decoder.weight', 'cls.predictions.transform.dense.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.




@LysandreJik
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.