Nevermind, this does exist, I was just getting an error because your first dimension for attn_mask needs to be batch_size * num_heads. I just had batch_size. This is explained in F.multi_head_attention_forward, but not in the MultiheadAttention forward method.