A usually simpler option that you have is to add this parameter to your model, usually on the __init__ method.
Doing something like:
self.my_param = ...
it will be automatically registered.
And then when you do something like
optim.SGD(model.parameters(),...)
it should work well.
Do notice that if you want to accumulate lists of parameters, you need to use ModuleList()