urkatz:

once the network is trained I want to use the front end encoder and decoder separately so that I can do things like explore other points in the latent space (kind of like a VAE), build another network that outputs new latent vectors that are meaningful (like a GAN), or build other backends that do things with the latent vector that has been “compressed” from an input somewhere else


Don’t you think this is like going too far for linear layers? In case you think about continuing with linear layers, you can use residual connections as in DenseNets or ResNets – try connecting blocks near the output with deeper blocks. It helps with the vanishing gradient problem and also helps to increase the model complexity in a nice way.
PS–never think you are too n00b to ask a simple question in deep learning. Many people mindlessly add layers. This just shows you do not