Transformers, Pytorch, Machine Learning, Deep Dives, Data Science.
attention_mask=None, key_padding_mask=None): """ q : tensor of shape (batch_size, query_sequence_length, hidden_dim) k : tensor of shape (batch_size, key_sequence_length, hidden_dim) v : tensor of shape (batch_size, key_sequence_length, hidden_dim) attention_mask : tensor of shape