I created a custom dataset to load frames from a video and corresponding labels and combined them together using ConcatDataset to create a dataloader.
I noticed that in the first epoch, the GPU is well used at ~90% usage with n_workers = 8. But from second epoch on the average usage drops and there is always a period of no usage, and the threads are not feeding the GPU fast enough.
I have a Ryzen 2700x CPU with RTX 2080Ti and loading data from an SSD.
I am wondering if this slowdown has something to do with the CPU (not being overclocked), in which case why does this slowdown not happen until the second epoch. I dont see this effect anytime during the first epoch.
Any help would be appreciated. Thanks.