So when I ran my code around 2 days ago it was taking 1 min per epoch, and when I ran exactly the same code today it was taking around 10 mins per epoch. In fact, it started very fast and after about 30-50% of the epoch is done, it suddenly slows down.
I noticed that my disk space (4tb hdd) had only about 70 gb left. So I cleared out around 150gb of data. After this, the training time again took around 1 min epoch.
Any idea why disk space is so much related to the training time?