Solved by Naman-ntc in post #6 
              
              
                @ptrblck I performed the tests. 
Surprisingly it doesn’t double up the memory, infact it a model that used 8.3 GB on forward pass took less than 9 GB on backward pass. It might have allocated space for gradients during forward pass only. 
Also just now noticed that Variables has been deprecated!!! W…