Solved by mortonjt in post #4 
              
              
                Final UPDATE : I think I’m able to fix the problem.  It boiled down to better understanding the pytorch-lightning semantics. 
First, the second_order_closure function is defined by wrapping the training_step and the backward step (see this line : https://github.com/PyTorchLightning/pytorch-lightning…