set the requires_grad attribute of vgg's parameters to False. The parameters will still be presented in the parameters of PerceptualLoss, but they won’t be optimized:
for param in self.vgg.parameters():
    param.requires_grad = False