NN_pruning module for Question Answering
Hi!

I am trying to run the launch_qa_sparse_single.py file from the question answering example from your nn_pruning library (https://github.com/huggingface/nn_pruning). I haven't changed anything from the original code and I get this error:

You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
***** Running training *****
  Num examples = 131754
  Num Epochs = 20
  Instantaneous batch size per device = 16
  Total train batch size (w. parallel, distributed & accumulation) = 16
  Gradient Accumulation steps = 1
  Total optimization steps = 164700
  0%|                                                                                       | 0/164700 [00:00<?, ?it/s]Traceback (most recent call last):
  File "question_answering/launch_qa_sparse_single.py", line 33, in <module>
    main()
  File "question_answering/launch_qa_sparse_single.py", line 23, in main
    qa.run()
  File "./question_answering/xp.py", line 324, in run
    self.train()
  File "./question_answering/xp.py", line 312, in train
    model_path= model_path
  File "/home/ines/NN_pruning/venv_nn_prun/lib/python3.7/site-packages/transformers/trainer.py", line 1120, in train
    tr_loss += self.training_step(model, inputs)
  File "/home/ines/NN_pruning/nn_pruning/nn_pruning/sparse_trainer.py", line 86, in training_step
    return super().training_step(*args, **kwargs)
  File "/home/ines/NN_pruning/venv_nn_prun/lib/python3.7/site-packages/transformers/trainer.py", line 1542, in training_step
    loss.backward()
  File "/home/ines/NN_pruning/venv_nn_prun/lib/python3.7/site-packages/torch/tensor.py", line 245, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/home/ines/NN_pruning/venv_nn_prun/lib/python3.7/site-packages/torch/autograd/__init__.py", line 147, in backward
    allow_unreachable=True, accumulate_grad=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.LongTensor [16]] is at version 3; expected version 2 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

I found several solutions to this problem on the internet but all the solutions I came accross with tell me to change something in the architecture of the model. Unfortunately here, we are using a Trainer from the transformers library so I don't really know how to fix this issue. Thank you for your help.

I am running this code with torch==1.8.1 and cuda=11.1.
Hi there, would you mind opening this issue in `nn_pruning` repo? Thanks.<|||||>This was bypassed in nn_pruning then fixed in transformers in https://github.com/huggingface/transformers/pull/12026 .