Thanks a lot for this proposal!!
There was a proposal for the role-based RPC design with some similar concepts from @Kiuk_Chung, we might be able to join forces there.
@iffiX Regarding the PR submission, we are grateful about the intent for contributing! Before making decisions on the commitment, may I ask how much bandwidth will you be able to allocate to this project? Since this can be a major new feature, most likely we will need to go through the formal design proposal review --> API review --> code review process and will also need examples (in pytorch/examples repo) and tutorials (in pytorch/tutorials repo) for it. This can become a very involved effort, so that we will need to make sure we have all the things we need to hit the finishing line.