That does work in the sense that all the random values are different, but reproducibility is no longer maintained. What I did set though was:
def worker_init_fn(worker_id):
     np.random.seed(np.random.get_state()[1][0] + worker_id)

if __name__ == '__main__':
    np.random.seed(1)
    torch.random.manual_seed(1)
    torch.manual_seed(1)
...

    for e in range(epochs):
        np.random.seed(e * num_workers)
...

where the seed is reset after each epoch to a new value based on the epoch and the number of workers. Then use the worker_init_fn to initialize each worker based on the seed for that given epoch. It isn’t the best work around, and does cause reproducibility issues when the number of workers vary, but so far its the best work around I got.