Thank you for your answer!
The network can use loss defined be the outputs only once, so it makes sense that during training it will be able to only have the current model in gpu memory. However, running the data through the model naively overflows the memory, meaning that it creates three copies of the network. How would one tell pytorch to only load one example (anchor, positive or negative) at a time and then produce loss by only having one copy of the model in memory?

data, target, data_pos, target_pos, data_neg, target_neg = data.cuda(), target.cuda(), data_pos.cuda(), target_pos.cuda(), data_neg.cuda(), target_neg.cuda()
data, target, data_pos, target_pos, data_neg, target_neg = Variable(data), Variable(target), Variable(data_pos), Variable(target_pos), Variable(data_neg), Variable(target_neg)


optimizer.zero_grad()
output = model(data)
output_pos = model(data_pos)
output_neg = model(data_neg)