Hi, I am trying to implement a model of which the loss function depends on a pair of training data.  Specifically, given dataset D={x_i}, the loss function will be E(f(x_i), f(x_j)), where f is the neural network model.  I wonder if there is an efficient way to generate a stream of random pairs {x_i, x_j} for training?
Currently I have two following approaches in mind:


preprocess D to get the complete pair set containing N(N-1)/2 pairs (N is number of data in D) and then feed them into the model.  The problem is the memory usage (even at preprocessing stage) scales with N^2 which is not tractable.


sample {i,j} index at runtime for each pair, and manually pass the index to D (without using DataLoader) to get data for each batch.  But it introduces significant overhead and can be inefficient.


I am wondering if there exists some sort of “generator” that could handle this sampling problem efficiently.  Any suggestion will be appreciated.  Thanks!