A better way to process extended_attention_mask in BertModel.forward()
In the `forward()` method of the `BertModel` (https://huggingface.co/transformers/_modules/transformers/modeling_bert.html#BertModel.forward), the `extended_attention_mask` is processed in the following way: 

> extended_attention_mask = (1.0 - extended_attention_mask) * -10000.0

I know this is to make an additive mask so that the unmasked positions will be unchanged (by adding 0) and the masked positions will be very small (by subtracting 10000). 

But I think it is better to achieve this goal by the following way: 

> extended_attention_mask = torch.log(extended_attention_mask)
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.