One common thing I have faced with Error: Out of Memory is having a lot of num_workers (>2). The more the number of workers in the data-loader the more memory it takes and I have seen the memory usage increases drastically.