You may need register_buffer().

  
    
    
    Use and Abuse of .register_buffer( ) 
  
  
    Hi, 
I have some trouble understanding the use of register_buffer(). 
I found just a little bit of explanation in the docs, mentioning “running_mean” in BatchNorm. 
My questions are: 

When should I register a buffer? For what sort of Variables and for which not?
Could someone provide me with a simple example and code snippet of using register_buffer()? 
[3.] At the moment, I’m running some tests on an implementation of a custom gradient, which I subsequently modify. record the gradients before …