Hi, I having a problem on long loading times from my dataset, so I insert prints to my getitem function inside dataset class ( of course inheriting from data.Dataset), this is not my first dataset class, this is not my project, but the output I got was pretty weird.
I did everything “by the book”, I’m iterating my dataset with:
    for i, (inputs, targets) in enumerate(data_loader):


During test and val.
My get Item is looking something like this:
def __getitem__(self, index):
      start_time = time.time()
      Actions
      end_time = time.time() - start_time
      print("loading:",json_path,",took:",end_time )

What I expected to see, let’s i have batch 1 and one thread, I expected to see 1 line of

loading: $json_path,took: $end_time

and one line of iteration results:

Epoch: [1][2844/26517] Time 0.103 (0.186)      Data 0.049 (0.131)      Loss 6.0703 (6.1430)    Acc 0.000 (0.002)

For my understanding, I’m reading 1 (batch 1) file from my dataset and processing, but instead, I got a much bigger amount of lines for each.
What am I missing? It’s important for me in order to understand my main problem.
Thanks.