I think I’ve found the issue. It appears that the instance of the optimiser needs to be
Optimiser = torch.nn.Adam(Model.(Layer to be trained).parameters())

and it seems that passing all parameters of the model to the optimiser instance would set the requires_grad attribute of all the layers to True. This means that one should only pass the parameters of the layers to be trained to their optimiser instance.