Thanks, will try Pandas, our training record is split into multiple files, each files is multiple lines of string, each line is JSON string. so a little bit more to work on JSON, but definitely try Pandas
BTW, regarding The  DataLoader  processes should be independent from the  DistributedDataParallel  processes., so let’s see we have 32 data loader workers, and we create 16 multiprocessing for our training, then is it 48 process running? or little bit complicated and can not simply add together?