If i have a pretrained autoencoder/decoder that i want to use, and i want to train a model that would use that decoder as its output stage, how do i train them properly? I have several questions:
Should i .detach() tensor before feeding it to decoder, so:
def forward(self, input):
  out = self.main(input)
  out = out.detach() # <- should i do this?
  out = self.decoder(out)
  return out

If not,should i set up optimizer to train only new model, or model+decoder to fit them together better?
So if my model has two modules: main and decoder should i do this:
optimizer = optim.Adam(model.main.parameters());
or this:
optimizer = optim.Adam(model.parameters());
Thanks