If I combine them correctly I get constant memory consumption independent of depth, which is different from checkpointing (I only need the resulting tensors to compute the input gradients, assuming add_out is None).

The fact that you return a tuple might be a problem. Can you just return your three tensors like  return res_pred, add_out, factor  ?

return res_pred, add_out, factor does not make any difference.