Transformers, Pytorch, Machine Learning, Deep Dives, Data Science.
tensor of shape (batch_size, num_heads, key_sequence_length, hidden_dim//num_heads) value : tensor of shape (batch_size, num_heads, key_sequence_length, hidden_dim//num_heads) attention_mask : tensor of shape (query_sequence_length, key_sequence_length) key_padding_mask : tensor of shape