How do you want to use it? You could create a new Variable with requires_grad=True and register it as a parameter with your module: http://pytorch.org/docs/master/_modules/torch/nn/modules/module.html#Module.register_parameter 35 .