Bertology example is probably broken
Hello!

I tried to run `run_bertology.py` in the example dir calling it with 

```
export TASK_NAME=CoLA
python ./run_bertology.py     --data_dir $GLUE_DIR/$TASK_NAME     
                              --model_name bert-base-uncased     
                              --task_name $TASK_NAME     
                              --max_seq_length 128     
                              --output_dir ./tmp/$TASK_NAME/     
                              --try_masking 
                              --metric_name mcc
```

But it fails with 

> Traceback (most recent call last):
>   File "./run_bertology.py", line 346, in <module>
>     main()
>   File "./run_bertology.py", line 327, in main
>     eval_data = load_and_cache_examples(args, args.task_name, tokenizer, evaluate=True)
>   File "/data/home/dfiocco/BERT/run_glue.py", line 245, in load_and_cache_examples
>     list(filter(None, args.model_name_or_path.split('/'))).pop(),
> AttributeError: 'Namespace' object has no attribute 'model_name_or_path'

One fix to that problem should be (?) replacing all occurrences of `model_name` with `model_name_or_path` in `run_bertology.py`. Still, even with that "patch" running the code gives

> Traceback (most recent call last):
>   File "./run_bertology.py", line 346, in <module>
>     main()
>   File "./run_bertology.py", line 341, in main
>     head_mask = mask_heads(args, model, eval_dataloader)
>   File "./run_bertology.py", line 175, in mask_heads
>     print_2d_tensor(head_mask)
> UnboundLocalError: local variable 'head_mask' referenced before assignment

Trying another task (MRPC) I get instead

> 
> Traceback (most recent call last):
>   File "./run_bertology.py", line 346, in <module>
>     main()
>   File "./run_bertology.py", line 341, in main
>     head_mask = mask_heads(args, model, eval_dataloader)
>   File "./run_bertology.py", line 169, in mask_heads
>     _, head_importance, preds, labels = compute_heads_importance(args, model, eval_dataloader, compute_entropy=False, head_mask=new_head_mask)
>   File "./run_bertology.py", line 97, in compute_heads_importance
>     head_importance += head_mask.grad.abs().detach()
> AttributeError: 'NoneType' object has no attribute 'abs'

Did anybody manage to run the Bertology example without hiccups?

 

Yes, this example is still work in progress. Hopefully, I can finish it before ACL (but not sure).<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
