Why is the output bias computed separately?
Hi !

Sorry if this is a dumb question, but I don't understand why is the bias [added separately](https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/pytorch_pretrained_bert/modeling.py#L379) to the decoder weights instead of using `self.decoder = nn.Linear(num_features, num_tokens, bias=True)`? Isn't it equivalent?
The code section you linked follows the original TensorFlow code: https://github.com/google-research/bert/blob/f39e881b169b9d53bea03d2d341b31707a6c052b/run_pretraining.py#L257<|||||>Exactly.