This is getting closer, but that conditional is still throwing me off. I’ll use the network described in your message.
criterion = nn.BCELoss()
encoder_net = Encoder(input_size, hidden_size, output_size)
classifier_net = Classifier(2 * output_size, hidden_size)  # I'm allocating room for 2 tensors of the same size!

for data, target in data_loader:
    data1, data2 = data

    # first we encode each datapoint into some latent representation
    output1 = encoder_net(data1)  # N x output_size
    output2 = encoder_net(data2)  # N x output_size

    # We'll then concatenate these together so that the classifier network can see/operate on them both at the same time
    # Otherwise the two datapoints are always treated independently by the network.
    output = torch.cat((output1, output2), dim=1)  # N x (output_size * 2)
    pred = classifier_net(output)  # N x 1, in the range of [0, 1].

    # I'm assume that 'target' is of dimension N that contains a 1 if data1 should be ranked higher. Otherwise a 0.
    loss = criterion(pred, target)
    
    # Optimization step
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

Your “target”: is what’s getting me confused here. We need to know if data1 or data2 should be ranked higher. In my example, I’m assuming that target contains this information. Let me know if this is closer to what you’re looking for