Transformers, Pytorch, Machine Learning, Deep Dives, Data Science.
value, attention_mask=None, key_padding_mask=None): """ query : tensor of shape (batch_size, num_heads, query_sequence_length, hidden_dim//num_heads) key : tensor of shape (batch_size, num_heads, key_sequence_length, hidden_dim//num_heads) value : tensor of shape (batch_size, num_heads,