Yes but that would defeat the purpose of autograd in providing easy backward pass.
The networks are selected in the forward pass based on some characteristics of the input.
I will have to repeat these computations to select corresponding optimisers for each input.
Having a function 2 that makes all grads None makes code simple 2