Hi, I have some problems with fine-tuning the last layer of a Neural network.
Code is as followed:
for layer in self.layers[0:-1]: 
        for param in self.model.fc_layers[layer].parameters():
            param.requires_grad = False
        
        y_prime = self.model(X)
        loss = self.model.criterion(y_prime, y)
        
        self.model.optimize.zero_grad()
        self.model.zero_grad()
        
        loss.backward(retain_graph=1)
        self.model.optimize.step()

Model is a Neural network.
It didn’t work as I fine-tune the last layer(re-train), I get the same result while I didn’t fix the parameters of any layers.
something is wrong here, but I can’t figure it out.