Solved by Usama_Hasan in post #4 
              
              
                So, Both the functions are quite the same except 

This will reduce memory usage and speed up computations but you won’t be able to backprop. 
Except everything is the same, second function has much better implementation. 
Also 

I don’t see any use of this.