Every time a variable is back propogated through, the gradient will be accumulated instead of being  replaced. (This makes it easier for rnn, because each module will be back propogated through several times.)