cindybrain:

The question is how can I share a cache among multiple subprocesses in the same node, when using the method torch.distributed.init_process_group rather than the method of multiprocessing.


Hey @cindybrain, there is an ongoing discussion on this topic. See: [RFC] TorchStore - A Shared-Memory Tensor Store · Issue #64932 · pytorch/pytorch · GitHub 6
Does the proposed solution address your use case?
cc @cbalioglu