I want to realise the distributed model parallelism with PyTorch, but I cannot find any example for this. I just find some distributed data parallelism example. My problem is that, if I divide a computing graph into two nodes, how can I still continue to use autograd to computing the gradients and update the weights?
Any help will be appreciate.