Reason for using einsum in xlnet?
## ❓ Questions & Help

Hello.

This might be a newbie question, so I apologize in advance.

While reading your implementation of xlnet, I ran into several usages of `torch.einsum` function.

example) `k_head_h = torch.einsum('ibh,hnd->ibnd', cat, self.k) `

After studying the definition of einsum, I came to a conclusion that the above statement is exactly like using a linear layer (without bias) (from dimension h to n*d), and then resizing the output to be ibnd.

So if I'm not wrong, is there any reason to prefer using `torch.einsum` over `nn.Linear`?

Is it related to performance issues?

I ran a simple test, and `nn.Linear` seems to be a bit faster than `torch.einsum`.

I would really appreciate your help.

Thank you.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
