@ptrblck, I have created a virtual environment in Anaconda, reinstalled PyTorch and rerun the script and using Jupyter with this new environment. I would say the effect is the same.
Although, while playing around I noticed the following. I am iterating (in a for-loop) the parameter num_workers, and before I was iterating starting from num_workers=0 and increasing it with next iterations. So it worked for num_workers=0 and then the training was getting frozen for next iterations.
This time I was iterating num_workers in a descending order, say, starting from 4 down to 0 (specifically, 4,2,1,0). And it works for the first iteration, i.e. for num_workers=4 and then it gets frozen at the next iteration.
Then I tried Spyder and seems there iterating through num_workers works, however, plenty of similar warning messages are being thrown into the console:
C:\Users\Admin\.conda\envs\pytorch_env\lib\site-packages\numpy\core\_asarray.py:83: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
  return array(a, dtype, copy=False, order=order)

I also tried Colab and everything works flawlessly (in terms of different values of num_workers being iterated in a for-loop).
Is there anything else I can do to get it working on my machine?