Solved by vainaijr in post #2 
              
              
                I think einsum could be used here 
x = torch.randn(2, 3, 5, 6)
y = torch.randn(2, 3, 6, 7)
z = torch.einsum('abcd, abde -> abce', x, y)
z.shape