Thanks so much for your reply,
self.memory keeps increasing indeed. self.memory is an Experience Replay buffer with a maximum size of 1e6, so I keep adding environment “observations” in it till it is full and then I keep updating it in a queue basis.
I finally found out that the step slowing time at every training epoch is when sampling the batch using the DataLoader. So I am know getting the data sampling directly from the buffer without using the DataLoader and things work far faster.
This slow performance by the DataLoader happens also when having a Dataset directly loaded from an hdf5 file (so its size doesn’t keep increasing with time).
Any insights what I am doing wrong?
Increasing num_workers doesn’t solve the issue since no part of my code is parallelized, so increasing num_workers won’t make any difference I guess.