I found a way that works in general with autograd when replacing parameters with some tensor for backpropagation through that tensor: instead of

self.hidden1.weight.copy_(U*W)


use

self.hidden1._parameters['weight'] = U*W


module._parameters[<name>] = <tensor>
 sidesteps nn.Parameter and, as far as I can tell, just treats the tensor normally with autograd.