I want to run my model on dataset and store all embeddings using DistributedDataParallel. I created dataloader with DistributedSampler and now want to store all embeddings in the form:
(image_name, embedding)
And after that I want to save them as csv or pickle file.
Will it be correct to create a global list and store data there or will there be conflicts with writing to the list?