TT_YY:

I have checked communication using args and global variables.
Non of them works for transmitting data from a GPU process to main(), when I use DDP.


This is true, because Python global vars are per-process concept.

Does everybody using DDP plot the loss charts in a spawned process on GPU?

This can be done using torch.multiprocessing.SimpleQueue. E.g., let the main process create the queue, pass it to the child process, and then let the child process put the loss object to the queue. Then, the main process should be able to see that.
The test below can serve as an example:

  
      github.com
  
  
    pytorch/pytorch/blob/2c4b4aa81bc8dba8272e9c7190edcaa3e114ec15/test/test_multiprocessing.py#L580-L600

def test_event_multiprocess(self):
    event = torch.cuda.Event(enable_timing=False, interprocess=True)
    self.assertTrue(event.query())

    ctx = mp.get_context('spawn')
    p2c = ctx.SimpleQueue()
    c2p = ctx.SimpleQueue()
    p = ctx.Process(
        target=TestMultiprocessing._test_event_multiprocess_child,
        args=(event, p2c, c2p))
    p.start()

    c2p.get()  # wait for until child process is ready
    torch.cuda._sleep(50000000)  # spin for about 50 ms
    event.record()
    p2c.put(0)  # notify child event is recorded

    self.assertFalse(event.query())
    c2p.get()  # wait for synchronization in child
    self.assertTrue(event.query())
    p.join()