Thank you, it means there is no api provided in torch.nn.MultiheadAttention to do this, right? If I want to use my own attention function I need to implement from scratch without using torch.nn.MultiheadAttention module.