I used an python multiprocessing Pool and imap() function in my Dataset init() function to accelerate featurization my input.But my code hangs when initialize the Pool(). Any idea? here are some pseudo code.
from multiprocessing import Pool
class MyDataset(torch.utils.data.Dataset):
    def __init__(input):
         p = Pool(5)
         feature = list(p.imap(input, featurization_func))

ps: The MyDataset was then loaded with torch.utils.data.Dataloader and number_workers is set by 4.
Help appreciated! Thanks in advance~