Future attention masking in GPT/GPT-2?
Based on my understanding, [this](https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/pytorch_pretrained_bert/modeling_gpt2.py#L288) is the place where future attention masking for the causal model happens.

If this is the case

- Why is it called `bias`? 
- Why is this in the core of `Attention` module and not passed as a `attention_mask` parameter similar to BERT?


Hi Shubham,
This is a legacy from the original Tensorflow code (https://github.com/openai/finetune-transformer-lm/blob/master/train.py#L64-L69).<|||||>Thanks for the link to the original reference. 