Masking in Bert
I am not able to grasp the concept of attention masking in bert or the other transformers.
<img width="790" alt="Screenshot 2020-05-01 at 7 56 06 PM" src="https://user-images.githubusercontent.com/45225143/80812561-d37a4a00-8be5-11ea-9da6-469b0d2d3f8f.png">
According to the documentation I tried to experiment it out.
Here it is clearly mentioned that the 1's in position are going to be masked and 0's are not.
So i tried to experiment it out and got this result.
<img width="510" alt="Screenshot 2020-05-01 at 8 00 02 PM" src="https://user-images.githubusercontent.com/45225143/80813016-d1fd5180-8be6-11ea-8fcc-c86943fc003f.png">
Here are my results the numbers on the left are 0 and right non zero.
I still dont get the whole picture of how this code fits into the whole picture.

TLDR
In general terms, if any one could explain me attention masking in the self attention part of the code in transformers and its variants(possibly with code) it would be great.
I don't fully understand the question, but love the color theme :)<|||||>> I don't fully understand the question, but love the color theme :)

Hey thanks😄its cobalt on vscode. Have edited the question with tldr pls let me know on it.<|||||>That being said, please don't use screenshots. They are hard to read (especially on phones) and make it impossible to copy-and-paste your code. Use [code blocks](https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks) instead. I also don't quite understand your question. Do you have a general question about how attention works? Please use [Stack Overflow](https://stackoverflow.com/) for this, as the template clearly mentions.<|||||>> That being said, please don't use screenshots. They are hard to read (especially on phones) and make it impossible to copy-and-paste your code. Use [code blocks](https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks) instead. I also don't quite understand your question. Do you have a general question about how attention works? Please use [Stack Overflow](https://stackoverflow.com/) for this, as the template clearly mentions.

Sure will follow the guidelines, closing the issue.