The  ResnetBase  and  self._lock  are not clear. Does the lock represents thread lock or something else?

Ah, thanks for the catch. Yep, this is a thread lock to prevent race. The full example code is here: https://github.com/pytorch/examples/blob/master/distributed/rpc/pipeline/main.py 5
Let me add the missing ResNetBase to the tutorial.