Thanks!
Notice that:


I’m not talking about using “managed memory” in the sense that is mentioned there - everything can still only use cudaMalloc. The eviction and reloading of variables can be done at high level in the forward/backward passes.


It will not kick-in until GPU memory is over, so it will be identical in performance to existing system until GPU mem is done.


Anyway, I guess that the burden of proof is on me then, thanks for sharing the relevant thread