I think the torch.nn.MultiheadAttention has a mask argument

forward (query , key , value , key_padding_mask=None , need_weights=True , attn_mask=None )

You can watch this video for an explanation Self Attention with torch.nn.MultiheadAttention Module - YouTube 14