Is it recommemded to use torch.cuda.empty_cache() before every training for every batch? Is it going to affect the performance ?
My model leads to OOM error for some batches becasue of its dynamic nature. So I am trying to emtpy_cache and garbage collect before every batch of data.