Based on the code snippet and your explanation it seems that the data loading is wrong.
The Siamese dataset returns:
return (img_0, label_0), (img_1, label_1)

so I assume that this unpacking is wrong:
    for batch_idx, (data, target) in enumerate(train_loader):
        target = target if len(target) > 0 else None
        if not type(data) in (tuple, list):
            data = (data,)
        if cuda:
            data = tuple(d.cuda() for d in data)
            if target is not None:
                target = (t.cuda() for t in target)

as it seems you are assigning the img_0 and label_0 as the data and img_1 and label_1 as the target.