Solved by dbp.pat94 in post #7 
              
              
                I think what you are trying to say is that if I want the nn.Parameters to “record” history, the example I talked about above uses register_buffer instead of nn.Parameter as a neat hack. 
I think it’s starting to make sense now. Here’s what I think is going on (code: Daniel’s code for ‘Learning To Re…