Hello. I have a script which processes given data and produces a pytorch binary output (.bin and .idx files). When I run this script in isolation everything works fine. But when I run multiple instances (using thread pool) of this script in parallel, I see that some of the output folders have missing .idx files. Any ideas why this could be happening ?
Also can we regenerate the .idx files based on the .bin files ?