Sure.
I am using the same model defined in this imagecaptioning repo 2
I am forward propagating the features

pred, att_w = model(fc_feats, bbox_feats, labels)
gradInput_gt = torch.autograd.grad(outputs=pred, inputs=(bbox_feats), grad_outputs=one_hot_gt.float(), create_graph=True)[0] # computing gradients wrt input features
gradInput_gt_sum = gradInput_gt.sum(dim=2)
criterion = nn.MSELoss()
loss = criterion(gradInput_gt_sum.float(), bbox_scores.float())
torch.autograd.grad(outputs=loss, inputs=(model.parameters()), create_graph=True, retain_graph=True, only_inputs=False, allow_unused=True)[0] # this works
loss.backward(retain_graph=True) # this throws the above mentioned error (once_differentiable) at runtime.

Hope this helps. Thanks