Hello, Thanks for your answer. I have some doubts still.
What was the motivation to choose this interpretation and choosing dim 4 as sequence dimension?
In Line 4, after you have squeezed H dimension in line 3, it is still mentioned in the comment, also, why C = T? Shouldn’t it be W = T?
Also, shouldn’t input to linear layer be in shape of 2D tensor? Shouldn’t we flatten it? You have passed a 3D tensor to linear layer. Will it work?