For anyone wondering, I solved this by using torch.multiprocessing.Manager
In main function
from torch.multiprocessing import Manager
from torch import multiprocessing as mp
from copy import deepcopy

with Manager() as manager:
    train_results = manager.list()
    # spawn
    mp.spawn(train_worker, nprocs=ngpus, args=(train_results))
    # copy out
    results = deepcopy(train_results)

# postprocess results to collect data

In train_worker:
def train_worker(tid, train_data):
    ...
    train_data.append((tid, epoch_num, loss,acc,time,num_correct))
    ...


Then after training you can use pandas to do some stats and collection of data.