I want to create a neural network generating the parameters of another NN. The NN ‘A’ generates ‘B’.
But there is a problem: the output of ‘A’ is a ‘tensor’ (with requires_grad = True) while the parameters of ‘B’ are ‘Parameter’. Since I want to backward the gradient up to the parameters of ‘A’, I am looking for a conversion tensor -> Parameter that keeps the computational graph.
A naive conversion ‘w = Parameter(y)’, where w is the generated parameter of ‘B’, and y is the output of ‘A’, does not work: the computational graph is cut between w and y (the gradient of the parameters of ‘A’ are None, while the gradient of w, which should be non-leaf, is not None).