@lesscomfortable this is strange. OK, can you please provide pytorch and ignite versions and confirm that you can not run this code snippet: Distributed training with CPU's - #4 by vfdev-5 ?
I created a colab to show that above code snippet is running correctly on CPUs: Google Colaboratory 3
If you could modify it to bring your code and execute it there (with some random data) and reproduce the issue, it would be nice.