NAN return from F.softmax function in pytorch implementation of BART self-attention
Pytorch 1.7.1 with GPU
transformers 3.0.2

Filling all masked positions with "-inf" may cause a NAN issue for softmax function returns.
It may cause similar issues in other models and other versions of the same model as well.<|||||>HI @KaiQiangSong 

We haven't yet observed `NaN`s with BART specifically, could you post a code snippet where the model returns `NaN` so we could take a look ?<|||||>> HI @KaiQiangSong
> 
> We haven't yet observed `NaN`s with BART specifically, could you post a code snippet where the model returns `NaN` so we could take a look ?

Sorry that, I couldn't publish my code now due to it is unpublished research.
I've fixed the issue myself with changing the mask_fill of float("-inf") to -1e5 (for supporting AMP as well).
Just post this issue here to let you know there might be a potential issue.<|||||>I have the same exact problem.

Ill try with the -1e5 trick and see if it helps me too.

Thanks a lot!<|||||>> I have the same exact problem.
> 
> Ill try with the -1e5 trick and see if it helps me too.
> 
> Thanks a lot!

glad that my solution helps.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.