I just want to make sure that I am understanding this part correctly. I see in the MultiheadAttention module (https://pytorch.org/docs/stable/_modules/torch/nn/modules/activation.html#MultiheadAttention 8) that if the $q, k, v$ tensors are all the same dimension, then the model calls self.register_parameter(<name>, None) which is the used later in the nn.functional.multi_head_attention_forward function.
If the registered parameters are None (or like a few lines before that nn.Parameter(torch.empty(<size>))) then that would mean that they are there, but they have no effect on the calculation, more like just a placeholder so they exist. Is this correct?