The full same code is: https://gist.github.com/knsong/1b2aeaf1cdfef28d52138df5d05cd949 2 , and my custom autograd Function is there, and if you execute it with  python backward_inplace.py , you will see the gradient error between the two version(inplace or no inplace) of the Function, like grad x error max: tensor(0.8613).