I’m currently using something like this:
for epoch in range(1):
    for img_path,emb_path in zip(img_emb_path,text_emb_path):
        train_dataset=CustomDataset(img_path,emb_path)
        train_data=DataLoader(train_dataset,batch_size=64,shuffle=True)
        for i, (inputs, labels) in tqdm(enumerate(train_data)):
            a=a+1
        print('singly npy processed')

My memory seems to be full with two runs of the iteration over DataLoader. Any suggestions for the same?