Hi,
I’m working with VAEs, 3 individual ones: vae1, vae2, vae3. In vae3 i’m loading the pre-trained encoder of vae2 and the pre-trained decoder of vae1. After the decoder i add some layer aswell.
My problem is now: I want to freeze the decoder. Therefore, just the encoder and the last layer should be trained.
I’ve read a lot about freezing layers but i’m still not sure if it is enough to run through the decoder (existing of Conv2d, ConvdTrans, BatchNorm, LeakyRelu) and just set require_grad=false… The optimizer is initialized with model.parameters(). Is this the proper way? Will the encoder and the last layer be correctly optimized?
Thank you!
EDIT: for clarification: the architecture is basically a variational autoencoder, which consists of a trainable encoder, a froze decoder and some trainable layer after the decoder.