Yes, I agree with all of you said except I do not test it on GPU (cuda:0, cuda:1) since without enough GPU devices.

So basically you mean the memory leak is reproducible as long as “cuda” is replaced by “cpu” in the tutorial for the ResNet.
If so, can you file a bug here. I will let RPC developers aware of this bug. Thanks!