Usually you would freeze the previous layers, which should not be trained, by setting their requires_grad attribute to False and I’m unsure how you are using the nn.ModuleList. Could you explain your current use case of fine tuning the last layers a bit more, please?