Given the code that I see I can only try to give some pointers that might help. Given that you posted in NLP and use embeddings, I assume that you work with text.


You’re unsqueezing your embeddings from 3d to 4d, so I assume that you use a nn.Conv2d. What are the kernel sizes you choose. Note that you could also use nn.Conv1d which would make the unsqueezing obsolete


To be honest, I have no idea what the expand() methods really does or why you need it. But any method that is needed to “fix” the shape of tensors needs to be applied with great care.


Your setup looks a bit like a Siamese Network. However, you use 2 different CNN layers and 2 different RNN layers – only 1 embedding layer though. In a Siamese setting, you would push both inputs through the same layer.


I trust that you define your LSTMs with batch_first=True 


Why do you transpose xA and xB and permute the first 2 dimensions after the embedding layer. I’m pretty sure you can simply remove both .t() and both permute() operations.