padded positions are ignored when embedding position ids
 When embedding position ids, all positions are considered. 
```
seq_length = input_ids.size(1)
position_ids = torch.arange(seq_length, dtype=torch.long, device=input_ids.device)
```
This is different from most transformer implementations. 
Should it be
```
position_ids = np.array([
        [pos_i+1 if w_i != PAD else 0
         for pos_i, w_i in enumerate(seq)] for seq in batch_seq])
```
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
