Solved by ptrblck in post #5 
              
              
                Forward hooks would work fine and I don’t think the hooks themselves create the increased memory usage. 
Based on the code snippet you’ve provided it seems you are executing the forward method sequentially using the child modules of the base model, which could yield the OOM error. 
If the model is w…