hi again Chris,
First of all, thanks a bunch for your tips.

As for your first hint, I used nn.Conv2d since I decided to add more channels for the next test.
*OMG, it’s a typo. EXPAND method was just a trial, I shouldn’t have sent it.
*Yeah, you are right. and also batch_first=True.
In respect of permuting and transposing, you’re definitely right. I wanted to exactly prepare the same shape as the torch example for the embedding layer. (I had thought that  it might be the problem)
However, having said that, I don’t know what is wrong. One more thing, I have a varied size of sentences so I used pad sequences to fix the size of each input (It was the same thing as previous papers). I couldn’t use pack_padded due to network producing higher Loss. perhaps it’s because of the differences in the length of two sentences as one input each time.