Solved by phan_phan in post #2 
              
              
                Hi, 

The TransformerEncoder “transforms” each input embeddings with the help of neighboring embeddings in the sequence, so it is normal that the output is homogeneous with the input : it should be the same shape as the input. 
You can look at the implementation of nn.TransformerEncoderLayer for mor…