From what I understand, x.retain_grad() should be preferred over using the register hook method. See this for a comparison of various approaches to saving gradients of intermediate variables:
  
      github.com
  
  
    rasbt/deeplearning-models/blob/master/pytorch_ipynb/mechanics/manual-gradients.ipynb 6
{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Deep Learning Models -- A collection of various deep learning architectures, models, and tips for TensorFlow and PyTorch in Jupyter Notebooks.\n",
    "- Author: Sebastian Raschka\n",
    "- GitHub Repository: https://github.com/rasbt/deeplearning-models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [


  This file has been truncated. show original