That is the tutorial I am currently following!  Because of this project requirements, I have to use the decoder part as well, since I will also have to probably train it on translation tasks.
What I would like to do now is plot heatmaps of the attention matrix activation like this:

2680×1956

I am accessing the self_attention and multihead like so:
attn_activation = model.transformer.encoder.layers[1].self_attn.out_proj(emb)

But I am a little lost on how that heatmap is mapped… what is the value of every point ij in the map?