Solved by richard in post #2 
              
              
                If I am understanding the problem correctly, this should be implementable using a single matrix multiply. 
Assume that y is a tensor of shape (n_head, hidden_size), and that each linear layer takes in hidden_size in-dims and returns output_size out-dims. This means that each y'_i has shape (output_s…