For example
param = list(model.parameters())
optimizer = torch.optim.SGD(param[0][:10,], lr = 0.01, momentum=0.9)

or
optimizer = torch.optim.SGD([list(torch.split(param[0],10,0))[0], lr = 0.01, momentum=0.9)

It returns ValueError: can’t optimize a non-leaf Variable
I find out that after I split the parameters, it become a non-leaf node
What I want is to dynamically train sub newtork. Like [requires_grad=True/False dynamically 10]
I want to split the parameter into even smaller parts. Is there a way to do that?