Solved by albanD in post #2 
              
              
                Hi, 
Large memory increase during the backward is expected as you need to have both the forward buffers, the gradient tensors, and the intermediary gradients being computed.