Nutsy:

and it seems that passing all parameters of the model to the optimiser instance would set the requires_grad attribute of all the layers to True .


No, that’s not the case as seen here:
# Setup
model = models.resnet18()
for name, param in model.named_parameters():
    print(name, param.requires_grad)

# Freeze fc params
model.fc.weight.requires_grad = False
model.fc.bias.requires_grad = False
for name, param in model.named_parameters():
    print(name, param.requires_grad)

# Create optimizer and check, if layers were "unfrozen"
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
for name, param in model.named_parameters():
    print(name, param.requires_grad)

Again, feel free to post an executable code snippet, which we could use to debug.