Could you please post the complete Keras and PyTorch code, if that’s possible?
If you cannot post the complete code, a small snippet of the encoder and decoder part will be sufficient.
The decoder should keep the dimensions it gets from the encoder.
I don’t really understand the issue, but a look at the code will most likely change that.