hey,
actually soon after this I switched to using Pytorch-lightning 28 and using it for all my setups regarding compute and data. This resolved the issue. Could have been due to some issue in manual setup of DDP within Pytorch at my end.  I didn’t investigate further. I would highly recommend using lightning and save yourself a lot of time you might spend in various parts of your setup.