Hi, I need some help trying to make my model pass through gradients properly.
In my model, I have a series of conv layers, then linear layers. After the linear layers spit out an 2x8x8 grid, I apply  torch.abs, perform a cumulative sum operation on the grid, and upsample the grid to the size of 2x128x128. Then, I perform a grid_sample() operation to the source image using the upsampled grid, and finally the loss is computed by comparing the original image to the deformed image. In addition I have a l1 regularization term in the loss function, applied to the initial 2x8x8 grid as originally outputted by linear layers.
But when I print the gradients through each of these consecutive operations, none of them has a gradient (although I set the retain_grad to true). Does anyone have suggestions as to what might be the problem here? Thanks!
aug_batch gradient: requires_grad(False) is_leaf(True) retains_grad(None) grad_fn(None) grad(None)
source_batch gradient: requires_grad(False) is_leaf(True) retains_grad(None) grad_fn(None) grad(None)
input_image gradient: requires_grad(False) is_leaf(True) retains_grad(None) grad_fn(None) grad(None)
X gradient1: requires_grad(False) is_leaf(True) retains_grad(None) grad_fn(None) grad(None)
X gradient2: requires_grad(True) is_leaf(False) retains_grad(True) grad_fn(<ReluBackward0 object at 0x7f612bf44a90>) grad(None)
X gradient3: requires_grad(True) is_leaf(False) retains_grad(True) grad_fn(<ViewBackward object at 0x7f612bf44a90>) grad(None)
X gradient4: requires_grad(True) is_leaf(False) retains_grad(True) grad_fn(<ReluBackward0 object at 0x7f612bf44390>) grad(None)
X gradient5: requires_grad(True) is_leaf(False) retains_grad(True) grad_fn(<AddmmBackward object at 0x7f612bf44fd0>) grad(None)
output_grid gradient: requires_grad(True) is_leaf(False) retains_grad(True) grad_fn(<AddmmBackward object at 0x7f612bf44790>) grad(None)
output_grid gradient after view(): requires_grad(True) is_leaf(False) retains_grad(None) grad_fn(<ViewBackward object at 0x7f612bf44450>) grad(None)
output_grid gradient after abs(): requires_grad(True) is_leaf(False) retains_grad(None) grad_fn(<AbsBackward object at 0x7f612bf44390>) grad(None)
integrated_grid after cumsum(): requires_grad(True) is_leaf(False) retains_grad(None) grad_fn(<PermuteBackward object at 0x7f612bf44790>) grad(None)
Upsampled_grid gradient: requires_grad(True) is_leaf(False) retains_grad(None) grad_fn(<PermuteBackward object at 0x7f612bf44b90>) grad(None)
Target_est gradient: requires_grad(True) is_leaf(False) retains_grad(None) grad_fn(<GridSampler2DBackward object at 0x7f612bf44fd0>) grad(None)
Warped_image gradient: requires_grad(True) is_leaf(False) retains_grad(None) grad_fn(<SqueezeBackward1 object at 0x7f612bf44a90>) grad(None)
Warp_field gradient: requires_grad(True) is_leaf(False) retains_grad(None) grad_fn(<CopySlices object at 0x7f612bf44390>) grad(None)
L_TV_Loss gradient: requires_grad(True) is_leaf(False) retains_grad(True) grad_fn(<AddBackward0 object at 0x7f612bf44690>) grad(None)
Total_loss gradient: requires_grad(True) is_leaf(False) retains_grad(None) grad_fn(<AddBackward0 object at 0x7f612bf44a90>) grad(None)