You still have an option to treat h_out as a by-product (as if additional detach() was applied afterwards) from F instead of computing second order derivatives. I have no idea if that would work with your code. Otherwise, i.e. for y_out → (x,h_in) gradient flow, it should be ok.
You may also be interested in forward mode AD 1, something that is in the works atm. Or maybe existing autograd.functional.jvp() would work better for your case.