Solved by tom in post #2 
              
              
                The typical way to do this (i.e. PyTorch uses it internally, too) is to pass it to the forward and store it as a ctx attribute (ctx.my_hyperparameter = ...) and read it as ctx.my_hyperparameterin the backward. Only input/output tensors want to go through saved tensors, so you should be safe here. 
B…