Is it possible to just use arbitrary differentiable/supported functions to create other functions without having to implement their backward as described in examples?
One thing I really liked about TF is how you can just create an arbitrary compute graph of differentiable pieces. It’s not obvious how to do that here unless I’m missing something?
Lets say I want to quickly implement a GELU: y = 0.5 * x * (1 + tanh(sqrt(2 / pi) * (x + 0.044715 * x^3))))
Can I just do that using the differentiable tanh() and pow() ? Or will I have to create a special class and describe backward()?