Is it possible to access and edit the graph on the go?
What I mean is the following:
Say, we have variables a and b and we can compute c as a function of a and b, i.e. c = f(a, b). For the forward propagation we need to know a, b and the function and for the backward we need the derivative of the function and then any partial derivatives coming that way.
What if, the function f was too expensive and instead of calculating c based on the function f, we had a way around, say function g. Now, we can’t use function g in our computations as g is not differentiable whereas f is. Is there any way to embed this output c (based on g) for the forward propagation but still use f during backward propagation?
I hope the explanation is clear enough. Let me know if it’s not!