Could you elaborate a little more on what you want?



 octopusyo:

\sum_i W_i * x_i


What is the size of W_i? Wouldn’t it be (H, H)? This doesn’t lead to the correct output size you want so that’s probably not the case. einsum probably does what you want, though: https://pytorch.org/docs/stable/torch.html?highlight=einsum#torch.einsum 1