I think torch.einsum can simply do that. But I’m not sure whether it’s efficient or not.
>>> A = torch.randn((3,4))
>>> B = torch.randn((16,4))
>>> C = torch.einsum("ij,bj->bij", (A, B))
>>> C.size()
torch.Size([16, 3, 4])
>>> C[0]-A*B[0]  # for testing the correctness
tensor([[0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.]])
>>> C[5]-A*B[5]
tensor([[0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.]])