Solved by ptrblck in post #2 
              
              
                Would splitting the Datasetsinto anImageDatasetandCoordDataset` work? 
You could then use the nested loop approach and keep the image constant in the inner loop. 
Pseudo code for the idea: 
image_loader = DataLoader(image_dataset)
coord_loader = DataLoader(coord_dataset)

for image in image_loader:
…