[ALBERT] : ValueError: Layer #1 (named "predictions") expects 11 weight(s), but the saved weights have 10 element(s).
## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....): ALBERT

Language I am using the model on (English, Chinese....): English

The problem arise when using:
* [ ] the official example scripts: (give details)
* [ ] my own modified scripts: (give details)

import tensorflow as tf
from transformers import *
#Download AlbertMaskedLM model
model = TFAlbertForMaskedLM.from_pretrained('albert-large-v2')

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details) Initial validation 

## To Reproduce

Steps to reproduce the behavior:
import tensorflow as tf
from transformers import *
#Download AlbertMaskedLM model
model = TFAlbertForMaskedLM.from_pretrained('albert-large-v2')
1. 
2.
3.

<!-- If you have a code sample, error messages, stack traces, please provide it here as well. -->
The code throws an error as follows :

100%|██████████| 484/484 [00:00<00:00, 271069.99B/s]
100%|██████████| 87059544/87059544 [00:03<00:00, 28448930.07B/s]
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-28-a2c768b76a32> in <module>()
----> 1 model = TFAlbertForMaskedLM.from_pretrained('albert-large-v2')

3 frames
/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    287         # 'by_name' allow us to do transfer learning by skipping/adding layers
    288         # see https://github.com/tensorflow/tensorflow/blob/00fad90125b18b80fe054de1055770cfb8fe4ba3/tensorflow/python/keras/engine/network.py#L1339-L1357
--> 289         model.load_weights(resolved_archive_file, by_name=True)
    290 
    291         ret = model(model.dummy_inputs, training=False)  # Make sure restore ops are run

/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training.py in load_weights(self, filepath, by_name)
    179         raise ValueError('Load weights is not yet supported with TPUStrategy '
    180                          'with steps_per_run greater than 1.')
--> 181     return super(Model, self).load_weights(filepath, by_name)
    182 
    183   @trackable.no_automatic_dependency_tracking

/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/network.py in load_weights(self, filepath, by_name)
   1173         f = f['model_weights']
   1174       if by_name:
-> 1175         saving.load_weights_from_hdf5_group_by_name(f, self.layers)
   1176       else:
   1177         saving.load_weights_from_hdf5_group(f, self.layers)

/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/saving/hdf5_format.py in load_weights_from_hdf5_group_by_name(f, layers)
    749                          '") expects ' + str(len(symbolic_weights)) +
    750                          ' weight(s), but the saved weights' + ' have ' +
--> 751                          str(len(weight_values)) + ' element(s).')
    752       # Set values.
    753       for i in range(len(weight_values)):

ValueError: Layer #1 (named "predictions") expects 11 weight(s), but the saved weights have 10 element(s).
## Expected behavior

<!-- A clear and concise description of what you expected to happen. -->
TFAlbertMaskedLM model can not be loaded from pre-trained
## Environment

* OS: Linux (Colab)
* Python version: 3.6
* PyTorch version: Tensorflow 2.0
* PyTorch Transformers version (or branch):
* Using GPU ? Yes
* Distributed of parallel setup ?
* Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->

cc @LysandreJik <|||||>It should be fixed now, thanks for raising an issue.<|||||>Thanks @LysandreJik for your prompt response. The issue mentioned above is resolved but I am getting an error in converting predicted IDs back to token using AlbertTokenizer. Here is the error that I am seeing (pred_index value below is 29324). Please advise or let me know if I should open another issue as original issue has been resolved.

TypeError                                 Traceback (most recent call last)
<ipython-input-26-0151f2884b58> in <module>()
----> 1 pred_token = tokenizer.convert_ids_to_tokens([pred_index])[0]
      2 print('Predicted token:', pred_token)

2 frames
/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils.py in convert_ids_to_tokens(self, ids, skip_special_tokens)
   1034                 tokens.append(self.added_tokens_decoder[index])
   1035             else:
-> 1036                 tokens.append(self._convert_id_to_token(index))
   1037         return tokens
   1038 

/usr/local/lib/python3.6/dist-packages/transformers/tokenization_albert.py in _convert_id_to_token(self, index, return_unicode)
    172     def _convert_id_to_token(self, index, return_unicode=True):
    173         """Converts an index (integer) in a token (string/unicode) using the vocab."""
--> 174         token = self.sp_model.IdToPiece(index)
    175         if six.PY2 and return_unicode and isinstance(token, str):
    176             token = token.decode('utf-8')

/usr/local/lib/python3.6/dist-packages/sentencepiece.py in IdToPiece(self, id)
    185 
    186     def IdToPiece(self, id):
--> 187         return _sentencepiece.SentencePieceProcessor_IdToPiece(self, id)
    188 
    189     def GetScore(self, id):

TypeError: in method 'SentencePieceProcessor_IdToPiece', argument 2 of type 'int'<|||||>Hmm, I have no issues running this code snippet:

```py
from transformers import AlbertTokenizer

tokenizer = AlbertTokenizer.from_pretrained("albert-large-v2")

print(tokenizer.convert_ids_to_tokens(29324))
# or
print(tokenizer.convert_ids_to_tokens([29324]))
```

Is there a way you could give us a short code sample that reproduces the problem, so that we may debug what's happening? Thank you.<|||||>@LysandreJik  thanks for your response. I figured out the issue. Below is the code which reproduces the issue. In the below code, 'pred_index' comes out as numpy.int64 and when placed in 'convert_ids_to_tokens' method, it throws the error mentioned above. If I convert it to an int then it works fine.

Here is the example code to reproduce the issue

# Encode a text inputs
text = "What is the fastest car in the world."
tokenized_text = tokenizer.tokenize(text)

#Get tokenizer 
tokenizer = AlbertTokenizer.from_pretrained('albert-base-v2')

#Lets mask 'world' and check if model can predict it
tokenized_text[7] = '[MASK]'

#Convert tokenized text to indexes
indexed_tokens = tokenizer.convert_tokens_to_ids(tokenized_text)

#Download AlbertMaskedLM model
model = TFAlbertForMaskedLM.from_pretrained('albert-base-v2')

#Prediction
inputs = tf.constant(indexed_tokens)[None,:]
outputs = model(inputs)

#Lets check the prediction at index 7 (in place of [MASK])
pred_index =  tf.argmax(outputs[0][0,7]).numpy()
pred_token = tokenizer.convert_ids_to_tokens([pred_index])[0]
print('Predicted token:', pred_token)<|||||>Please note that above code works as is for BERT (but throws an error for Albert).<|||||>This is probably the exact same problem than https://github.com/huggingface/transformers/issues/945

If I understand correctly SentencePiece doesn't like numpy integers and crashes. Should we cast it to an int @thomwolf?<|||||>Yes I think so. We can probably just add a `int(idx)` in the base tokenizer class `PretrainedTokenizer` before the call to `_convert_id_to_tokens` so we can even input tensors in addition to np arrays.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
