ALBERT: The following keyword arguments are not supported by this model: ['cls_index', 'p_mask', 'is_impossible'].
## Environment info
- `transformers` version:  4.5.1
- Platform: Google Colab
- Python version: 3.7
- PyTorch version (GPU?): NA
- Tensorflow version (GPU?): 2.4.1
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help: 
@LysandreJik for albert issue
@sgugger for trainer (tf_trainer) issue

Models:/transformers/models/albert/modeling_tf_albert.py: @LysandreJik 
Library: /transformers/trainer_tf.py: @sgugger 


## Information
Model I am using albert (albert-base-v2) for SQuAD 2.0 Question Answering task fine tuning exercise:

The tasks I am working on is:
- I am using functions from official transformer scripts only with an official SQUaD 2.0 dataset task: (give the name)
https://colab.research.google.com/drive/13_ZEQJa_SNMTUh1OkOL1UfkWwObpGY2i?usp=sharing

[transformer_fine_tune_pre_trained_model_tftpu.zip](https://github.com/huggingface/transformers/files/6371487/transformer_fine_tune_pre_trained_model_tftpu.zip)

## To reproduce
Steps to reproduce the behavior:
1. Run the above script shared with GPU on Google Colab and it will give error when it tries to run trainer.train() with SQuAD 2.0 TF dataset.

## Error message:

ValueError: in user code:

    /usr/local/lib/python3.7/dist-packages/transformers/trainer_tf.py:697 distributed_training_steps  *
        self.args.strategy.run(self.apply_gradients, inputs)
    /usr/local/lib/python3.7/dist-packages/transformers/trainer_tf.py:639 apply_gradients  *
        gradients = self.training_step(features, labels, nb_instances_in_global_batch)
    /usr/local/lib/python3.7/dist-packages/transformers/trainer_tf.py:622 training_step  *
        per_example_loss, _ = self.run_model(features, labels, True)
    /usr/local/lib/python3.7/dist-packages/transformers/trainer_tf.py:742 run_model  *
        outputs = self.model(features, training=training, **labels)[:2]
    /usr/local/lib/python3.7/dist-packages/transformers/models/albert/modeling_tf_albert.py:1341 call  *
        inputs = input_processing(
    /usr/local/lib/python3.7/dist-packages/transformers/modeling_tf_utils.py:351 input_processing  *
        raise ValueError(

    ValueError: The following keyword arguments are not supported by this model: ['cls_index', 'p_mask', 'is_impossible'].


## Expected behavior
- 'squad_convert_examples_to_features' function should have taken care of what all features to be passed to the model based on pre-trained tokenizer defined.
- If not then what should be steps to remove/add features required for albert model training/fine-tuning tasks should be documented somewhere [As per my knowledge it is not available anywhere]
The function prepares the dataset for the standard models as well the XLNet model that requires more arguments. You will need to drop those columns for an Albert models.

We are in the process of reworking the TensorFlow examples, so there should be one clearer example for QA soon!<|||||>Thanks @sgugger ...
I tried to modify `squad_convert_examples_to_features` as in attached file to return without these columns but encountered anther error as below:
[transformer_fine_tune_pre_trained_model_tftpu (1).zip](https://github.com/huggingface/transformers/files/6379976/transformer_fine_tune_pre_trained_model_tftpu.1.zip)





##ERROR:
ValueError                                Traceback (most recent call last)
<ipython-input-59-d85abec8ae26> in <module>()
      1 # Training
      2 if training_args.do_train:
----> 3   trainer.train()
      4   trainer.save_model()
      5   tokenizer.save_pretrained(training_args.output_dir)

10 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
    975           except Exception as e:  # pylint:disable=broad-except
    976             if hasattr(e, "ag_error_metadata"):
--> 977               raise e.ag_error_metadata.to_exception(e)
    978             else:
    979               raise

ValueError: in user code:

    /usr/local/lib/python3.7/dist-packages/transformers/trainer_tf.py:697 distributed_training_steps  *
        self.args.strategy.run(self.apply_gradients, inputs)
    /usr/local/lib/python3.7/dist-packages/transformers/trainer_tf.py:641 apply_gradients  *
        self.optimizer.apply_gradients(list(zip(gradients, self.model.trainable_variables)))
    /usr/local/lib/python3.7/dist-packages/transformers/optimization_tf.py:232 apply_gradients  *
        return super(AdamWeightDecay, self).apply_gradients(zip(grads, tvars), name=name, **kwargs)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:604 apply_gradients  **
        self._create_all_weights(var_list)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:783 _create_all_weights
        self._create_slots(var_list)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/optimizer_v2/adam.py:127 _create_slots
        self.add_slot(var, 'm')
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:844 add_slot
        .format(strategy, var))

    ValueError: Trying to create optimizer slot variable under the scope for tf.distribute.Strategy (<tensorflow.python.distribute.one_device_strategy.OneDeviceStrategy object at 0x7f12026a45d0>), which is different from the scope used for the original variable (<tf.Variable 'tf_albert_for_question_answering/albert/embeddings/word_embeddings/weight:0' shape=(30000, 128) dtype=float32, numpy=
    array([[ 0.01270407,  0.05987824, -0.06812993, ..., -0.01226719,
             0.00817283,  0.00785217],
           [ 0.00733008, -0.00101211, -0.01069043, ..., -0.00968418,
            -0.0400394 , -0.04233308],
           [-0.02059615,  0.007892  ,  0.02363562, ...,  0.01533034,
            -0.00429517, -0.01246009],
           ...,
           [ 0.0135935 ,  0.00349383,  0.01223597, ..., -0.05456466,
             0.09235671, -0.05717891],
           [-0.00492554, -0.05208753, -0.00323149, ...,  0.03003517,
             0.0196551 ,  0.06015572],
           [ 0.03892251, -0.024089  , -0.01364627, ...,  0.04010094,
             0.05124779, -0.03588157]], dtype=float32)>). Make sure the slot variables are created under the same strategy scope. This may happen if you're restoring from a checkpoint outside the scope<|||||>Can someone please help..? I am stuck here<|||||>I have managed to remove extra tokens from the dataset but then TFTrainer.train() got stuck in infinite loop without any errors or logs.

Below is the link with latest code. Please suggest.
https://colab.research.google.com/drive/17Rx2rkiqag6YAz_FnU9HyHYtqHJgpNs0?usp=sharing<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.