@codeflux backward propagation ( gradient update ) is done everytime you pass a new variable. retain_graph just ensures that the computation graph is not cleared and retains its values. This helps us in performing backward propagation once more