If the forward computation of  f  is too expensive, why is it’s backward feasible? If you use the autograd, the backward will have a similar runtime. You have a simple formula for it?

F is expensive (both computationally and memory wise) but it’s not that it’s intractable. Plus, I can’t use g for backpropagation as it’s not differentiable.
Is it possible to manually replace the derivative of dg with df?
One the one hand I have bilinear sampling which is our f and the other hand since I know that in my case bilinear sampling will only be used for cropping, I can manually crop the region (which I denoted as g).
However, since I want the gradient to go through to allow the model to learn how to crop on its own, I have to use a differentiable function. Note that there are a few more details that make bilinear samping much more expensive than straightforward cropping.