A cleaner solution would be:
out = F.relu(self.bn1(self.conv1(x)))
with torch.no_grad():
    out = self.CE(out)

However, I see that you are using the CE layer several times, and this solution is not feasible since gradient will not flow through your second CE layer, and you will not be able to train self.layer1, therefore, I suggest you either:


manually clear the .grad property and set them to None by iterating on self.CE.parameters() after loss.backward() and before calling optimizer.step()


restrict the range of parameters passed to the optimizer, pesudo code is:


optimizer([p for p in your_model.parameters() if p not in your_model.ce.parameters()],
                lr=learning_rate)

These two ways will not create problems stated in your question.
I am not sure whether setting “requires_grad” to False will interfere with the gradient calculation, may be do a simple experiment with two linear layers yourself?