Solved by ptrblck in post #2 
              
              
                I think your way to permute the dimensions might not work, if you provide a single sample with a batch size of 1. 
Currently you are squeezing and unsqueezing the output of your embedding to change the dimensions to [seqlen, batch, input]. 
However, if your batch size is only 1, you’ll squeeze this …