Sounds to me like some OS memory management issue (ie data swapping to/from disk).  The fact that you have several hundred GB of free RAM during training is strange, though.  How are you measuring the free RAM?
To sandbox the issue, do you observe similar symptoms when you don’t use Amp and/or DataParallel at all?