If your current optimizer doesn’t have access to some parameters, it won’t be able to update them.
However, gradients would still be accumulated in these “unused” parameters, so if you decide after a while to create a new optimizer and pass all parameters, your training might blow up.