sorry I mistakenly submitted a issue twice. Plz ignore (help delete) this one. 
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
- `transformers` version: 3.2.0
- Platform: Linux-4.4.0-1111-aws-x86_64-with-debian-stretch-sid
- Python version: 3.6.8
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
@stas00 


 

## Information

Model I am using (Bert, XLNet ...): BART 

The problem arises when using:
*--pt16 --fp16_opt_level O1 + conda-forge apex w/ --fp16 --amp_backend='apex' to run finetune.py 

Got this error: 

  File "/home/ubuntu/anaconda3/envs/nightly/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 429, in fit
    self.accelerator_backend.setup(model)
  File "/home/ubuntu/anaconda3/envs/nightly/lib/python3.6/site-packages/pytorch_lightning/accelerators/gpu_accelerator.py", line 53, in setup
    model = self.trainer.precision_connector.connect(model)
  File "/home/ubuntu/anaconda3/envs/nightly/lib/python3.6/site-packages/pytorch_lightning/trainer/connectors/precision_connector.py", line 78, in connect
    model, optimizers = self.backend.connect(model, self.trainer.optimizers)
  File "/home/ubuntu/anaconda3/envs/nightly/lib/python3.6/site-packages/pytorch_lightning/plugins/apex.py", line 38, in connect
    self.trainer.reinit_scheduler_properties(optimizers, self.trainer.lr_schedulers)
  File "/home/ubuntu/anaconda3/envs/nightly/lib/python3.6/site-packages/pytorch_lightning/trainer/optimizers.py", line 143, in reinit_scheduler_properties
    scheduler.__class__.__mro__[idx].__init__(scheduler, optimizer)
  File "/home/ubuntu/anaconda3/envs/nightly/lib/python3.6/site-packages/torch/optim/lr_scheduler.py", line 74, in __init__
    self.optimizer.step = with_counter(self.optimizer.step)
  File "/home/ubuntu/anaconda3/envs/nightly/lib/python3.6/site-packages/torch/optim/lr_scheduler.py", line 56, in with_counter
    instance_ref = weakref.ref(method.__self__)
AttributeError: 'function' object has no attribute '__self__'



This is the same error reported in one of the reply in https://github.com/huggingface/transformers/issues/8403... 

The tasks I am working on is:
* [ ] to finetune BART on XSUM. 

