Sorry I have fixed it. It was rather embarrasing:
It should be
for name,param  in self.inception.parameters():

instead of
for name,param  in self.encoderCNN.inception.parameters():

and I shouldn’t do require_grad for each forward pass