This makes sense but I’ve seen a lot of code online that uses the non public API and if I’m not mistaken I’ve seen also some pytorch examples uses it as well.

Can you link such example please?

On another note, if you wanted to loop over the params of a module, do some operation and then remove that param, how would you do that with module.parameters?

The thing is that the internal API is not recursive (won’t look for parameters in child Modules). So they will give you very different result if you’re not careful!
The right way to do this is to use named_parameter() that will return the name of the attribute as well as the parameter and you can then delattr(mod, name) if you want to remove it.