No, that does not fix the problem.
For example, applying the worker_init_fn changes and touching nothing else my results are:
epoch:  0
7692698082559361259 0 37
7692698082559361259 0 12
7692698082559361260 1 40
7692698082559361260 1 15
epoch:  1
4064269471072392264 0 37
4064269471072392264 0 12
4064269471072392265 1 40
4064269471072392265 1 15
epoch:  2
2109959069025161 0 37
2109959069025161 0 12
2109959069025162 1 40
2109959069025162 1 15
epoch:  3
5577051506714156677 0 37
5577051506714156677 0 12
5577051506714156678 1 40
5577051506714156678 1 15
epoch:  4
2707168392203228096 0 37
2707168392203228096 0 12
2707168392203228097 1 40
2707168392203228097 1 15

What is very strange is that depending on the environment I use I may or may not see if the issue. I have my work set-up where I have a local environment on my local PC and a remote environment on my work sever and I have it all synced with PyCharm’s deployment set-up. When running the same original code I posted on my local PC, I don’t see the issue. But when I run it on the server, I see it. The environments are different and have different OS (ie, Windows 10 vs Centos 7), but I find the differences very strange. =