Movement Pruning does not achieve expected results
Transformer version: master or 4.6.1
tensorflow-gpu == 2.3.1
pytorch == 1.7.1
flax == 0.3.4

I tried to reproduce the results based on the script given in examples/research_projects/movement-pruning. 
The experimental results I got are not as good as the paper shows:

06/01/2021 22:55:40 - INFO - __main__ -   ***** Running evaluation  *****
06/01/2021 22:55:40 - INFO - __main__ -     Num examples = 10833
06/01/2021 22:55:40 - INFO - __main__ -     Batch size = 32
Evaluating: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 339/339 [01:42<00:00,  3.32it/s]
06/01/2021 22:57:22 - INFO - __main__ -     Evaluation done in total 102.215492 secs (0.009436 sec per example)
06/01/2021 22:58:27 - INFO - __main__ -   Results: {'exact': 0.33112582781456956, 'f1': 7.122522334334856, 'total': 10570, 'HasAns_exact': 0.33112582781456956, 'HasAns_f1': 7.122522334334856, 'HasAns_total': 10570, 'best_exact': 0.33112582781456956, 'best_exact_thresh': 0.0, 'best_f1': 7.122522334334856, 'best_f1_thresh': 0.0}

I almost followed each step as the 'README.md' says. The only difference I modified is 'BertLayerNorm = torch.nn.LayerNorm' according to [this](https://github.com/huggingface/transformers/issues/10892).

I also noticed the requirements in the [setup ](https://github.com/huggingface/transformers/tree/master/examples/research_projects/movement-pruning#setup) step. 

I wonder what should I do to reproduce the same results from the master branch.

Thank you. @VictorSanh 
that's very low of a score... i imagine it''s on squad.
could you give me more details about the command you are running?<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>I have the same problem. The accuracy is quite low when performing movement pruning. Have you solved it? 