Yeah that’s what I’m doing - but it didn’t work at validation time since there’s no gradient. I added with torch.enable_grad() for that case for validation time and it runs ! but I have an issue with part of that.
Let’s say I then do h - h.grad. That’s an operation that depends on h, that should be counted in the backpropagation, right ? But in this case it would be adding a gradientless, independent of h vector and so wouldn’t change anything. Now this is why I’d like to have directly access to the derivative of F.interpolate; since gradient flows through it, is there any way to call the internal backward function that must exist somewhere within the C code ?
Again - what kind of filter does pytorch downsampling use ? Assuming it’s standard downsampling, I would be able to write a derivative for it with simple operations.