I have to admit that I don’t quite understand what you’re doing. Any chance you could share your code?
In my approach I essentially start with the [PyTorch Seq2Seq Tutorial] (just without attention)(https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html 1), where input and target are the same sequence instead of different languages. To extend it to an autoencoder, I’ve made the following extensions

Encoder:

Flatten hidden state from (num_layers*num_directions, batch_size, hidden_dim) to (batch_size, num_directions*num_layers*hidden_dim)

Push flattened hidden state to one or more linear layers to reduce dimension


Decoder

Push flattened hidden state to one or more linear layers to increase dimension
Unflatten hidden state from (batch_size, num_directions*num_layers*hidden_dim) to (num_layers*num_directions, batch_size, hidden_dim)




The code can be viewed in my Github repository 2.
I really don’t know if this the right way to go. If I only flatten and unflatten the hidden state (i.e., without for linear layers) it essentially becomes the Seq2Seq tutorial. At least I can make sure that I do the flatten/unflatten correctly.