Fine tuning time did not change much after freezing layers 
Hi, 

I am using simple_lm_finetuning.py to fine tune the model. I wanted to freeze all parameters from the very beginning to the beginning of the 12th transformer layer, I looked into parameters by name and used a counter, took the value of the counter corresponding to the beginning of the 12th layer, and used this value to freeze all layers before 12th layer. You can understand better from this piece of code I added to simple_lm_finetuning.py. 

```
ctr = 0 
for name, param in model.named_parameters():
    ctr += 1
    #print(ctr)
    #print(name)
    if(ctr < 183): #183 is where 12th transformer layer starts
        param.requires_grad = False
```

With Nvidia K80 and original simple_lm_finetuning.py, one epoch required about 37-38 hours. After adding this piece of code, it required about 28 hours. Since I have frozen all parameters from the beginning to the 12th layer, I was expecting more reduced time. Where am I wrong?

I am open to other suggestions of fine tuning methods that requires less computational time. 
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
