Pooler weights not being updated for Multiple Choice models? 
I'm trying use pretrained BERT to finetune on a multiple choice dataset.

The parameters from `pooler` are excluded from the optimizer params [here](https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/pytorch_pretrained_bert/modeling.py#L1044-L1047), however, the MutlipleChoice model does indeed use `pooled_output` (which passes through the `pooler`) [here](https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/examples/run_swag.py#L392).

I wasn't able to find a similar exclusion of `pooler` params from the optimizer in the official repo. I think I'm missing something here. Thanks for your patience.
Indeed this looks like a bug in the `run_swag.py` example.
What do you think @rodgzilla?
Isn't the exclusion of the pooler parameters from optimization ([line 392 of `run_swag.py`](https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/examples/run_swag.py#L392)) a typo?<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>@thomwolf -  This surely looks like a bug, should I just send a hotfix and close it?<|||||>Yes!<|||||>Fixed in #675.