Minor redundancy in model defintion?
this is a _major_ nitpick but it was a bit confusing at first:

https://github.com/huggingface/pytorch-pretrained-BERT/blob/822915142b2f201c0b01acd7cffe1b05994d2d82/pytorch_pretrained_bert/modeling.py#L206-L212

L212 can simply be replaced by `self.all_head_size = config.hidden_size` as you already error out if the result of division isn't a whole number. 
Yes, feel free to submit a PR. Otherwise, I'll fix it in the next release.