The output is computed using new weights before the lines you referenced here https://gist.github.com/SsnL/f2f56534aefb22d8612dbd7a5da28ed8#file-reparam_module-py-L51 6. I am setting back to old ones because I didn’t want the module weight to be changed before and after forward_with_weights