Any idea about that, or do you think that it’s still an implementation detail depending on the optimizer?

It definitely is. Other optimizers will just plain error out if you try to add the parameters after they are initialized. So 100% implementation detail