If you are using shuffle=True as it seems to be the case, the target tensors will most likely be different.
I misunderstood your use case, as I thought you would like to extend the dataset and call the additional samples in a sequential way.
Could you explain the correspondence between both datasets, i.e. is train_set[0] corresponding to train_set_2[0]?
If so, I would suggest to implement a custom Dataset and return the pair for a single index.
Here is a small example:


class MyDataset(Dataset):
    def __init__(self, dataset1, dataset2):
        self.dataset1 = dataset1
        self.dataset2 = dataset2
        
    def __getitem__(self, index):
        x1, y1 = self.dataset1[index]
        x2, y2 = self.dataset2[index]
        
        if (y1 != y2).any():
            raise RuntimeError('ERROR! Target mismatch')
            
        return x1, x2, y1
        
    def __len__(self):
        return len(self.dataset1)  # Assuming both have same length