Hi,
The backward engine in pytorch only allows you to perform L-op using backward mode AD.
R-op is tricky for neural nets when using backward mode AD because you don’t want to compute the full Jacobian for memory size reasons. You would need forward mode AD to implement that efficiently which is not implemented.
I am not sure how hard it would be to do a forward mode AD engine for pytorch. But definitely non trivial.