Thanks for the reply! The latter option would probably work.
I tried using a Variable, but the tricky thing is that a Variable in a module won’t respond to the cuda() call (Variable doesn’t show up in the parameter list, so calling model.cuda() does not transfer the Variable to GPU). Note that overriding the cuda() method won’t work either.
I found a better way to solve this problem.
we can still use nn.Parameter, but exclude it from the parameter list by overriding the named_parameters() method:
def named_parameters(self, memo=None, prefix=''):
    return iter(())

Overriding parameters() won’t work because only named_parameters() is being called recursively (if you want to use the customized module as a layer in a network).