Fix Llama's attention map handling for left padding which causes numerical instability and performance drops
Hi this PR is trying to address the performance drop and potential numerical instability caused by vanilla left padding in Llama.
Here is the explanation:
1. If we initialize the tokenizer with left padding and call model.generate without passing in corresponding attention_mask, the code will run, but for the instances who are left padded, its unpadded tokens will "see" the padded tokens. This will cause performance drop a lot ! At least in my case, my performance of llama2 in socialQA drops from 55% to around 20% if I use left padded batch inference instead of one by one generate.
2. If instead, I passed in the attention_map generated by the left_padding tokenizer to model.generate function, the model will throw an error when doing sampling because some values in the hidden states are inf or nan. This numerical instability suddenly appeared because train-test mismatch: **By examining the locations of these infs/nans, I found them only shows up in the position of those padded token and are caused by the attention_map.** 
3. The reason why attention map are causing the numerical instability is because the current way of generating attention mask did not considered the left padded situation and it will cause the left padded tokens to have a fully masked attention tensor ! While the model was never trained with any token that can not see any(including itself) token, the model thus generates anomaly values and creates nan/inf. 

So this PR is trying to fix two bugs I observed:
1. The attention_mask created for left_padded values will contain -inf value due to the operation "expanded_attn_mask + combined_attention_mask".  Consider the attention_map that looks like this ([[1, 1, 1, 1, 1], [0, 0, 0, 1, 1]]). The combined_attention_mask created by line 585 will look like this (under float16)
```
tensor([[[[     0., -65504., -65504., -65504., -65504.],
          [     0.,      0., -65504., -65504., -65504.],
          [     0.,      0.,      0., -65504., -65504.],
          [     0.,      0.,      0.,      0., -65504.],
          [     0.,      0.,      0.,      0.,      0.]]],


        [[[     0., -65504., -65504., -65504., -65504.],
          [     0.,      0., -65504., -65504., -65504.],
          [     0.,      0.,      0., -65504., -65504.],
          [     0.,      0.,      0.,      0., -65504.],
          [     0.,      0.,      0.,      0.,      0.]]]], device='cuda:0',
       dtype=torch.float16)
```
and the expanded_attn_mask created will look like this 
```
tensor([[[[     0.,      0.,      0.,      0.,      0.],
          [     0.,      0.,      0.,      0.,      0.],
          [     0.,      0.,      0.,      0.,      0.],
          [     0.,      0.,      0.,      0.,      0.],
          [     0.,      0.,      0.,      0.,      0.]]],


        [[[-65504., -65504., -65504.,      0.,      0.],
          [-65504., -65504., -65504.,      0.,      0.],
          [-65504., -65504., -65504.,      0.,      0.],
          [-65504., -65504., -65504.,      0.,      0.],
          [-65504., -65504., -65504.,      0.,      0.]]]], device='cuda:0',
       dtype=torch.float16)
```
And in line 598 these two variables are added together. I believe it will be now clear why left padding causes the attention_map itself contains -inf values and why some tokens has a fully masked attn tensor.
3.  My solution then is straightforward, I clamped the variables so it does not overflow, and I forces the left padded values to at least attend to itself. Though the hidden states of the left padded values will not be used by the unpadded tokens due to the attention map, making it cleaned of inf/nan will not break the generation process. 
4. I tested in my local cases and I did not observe any performance drop or nan errors during sampling. Though I am not sure if my patches will break any other use cases. 



cc @ArthurZucker 