SpanBert always predicts the same token
I tried to use this implementation - https://huggingface.co/SpanBERT/spanbert-base-cased, but as a prediction I always get the same exect outoput no matter where I put [MASK] in a sentence. Here is the code.  Am I doing something wrong?

    import torch
    from transformers import AutoTokenizer,AutoModel
    tokenizer = AutoTokenizer.from_pretrained("SpanBERT/spanbert-base-cased") 
    model = AutoModel.from_pretrained("SpanBERT/spanbert-base-cased")
    model.eval()
    model.to('cuda')

    text = "[CLS] Who was Jim Henson ? [SEP] Jim Henson was a puppeteer [SEP]"
    tokenized_text = tokenizer.tokenize(text)
    print("tokenized_text",tokenized_text)

    themask = 2
    tokenized_text[themask] = '[MASK]'

    indexes = tokenizer.convert_tokens_to_ids(tokenized_text)
    indexes_tensor = torch.tensor([indexes])
    indexes_tensor = indexes_tensor.to('cuda')


    with torch.no_grad():
        outputs = model(indexes_tensor)
        predictions0 = outputs[0]


    the_index = torch.argmax(predictions0[0, themask]).item()
    theresult = tokenizer.convert_ids_to_tokens([the_index])[0]
    print("theresult",theresult)
    print("the_index",the_index)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
