Solved by ptrblck in post #2 
              
              
                This error might be raised, if you were using multiple processes (e.g. via DistributedDataParallel) and didn’t guard the storing of the checkpoint against multiple writers. 
You could use the ImageNet example to only use the first process to store the checkpoint.