I don’t really understand the proper way if there is one, on how to progressively add layers neural networks for something like progressive autoencoder or gan.  First of all, should you create the entire network first and block access to the bigger input layers and just grab results from the inner layers first? Or do you make a small network first and when it hit some threshold we would add a new layer to the network? Finally, how do we add new layers to the network during training? I would like to experiment with a progressive autoencoder, for input its easy we can just transforms.resize() but how do you add outer network layers that takes new inputs on the fly?
Edited: Also if its possible to add new layers during training do I need to call model = model() again? Will that reset all the previous weight parameters?