In that case you could pass both image tensors to your forward method as:
    def forward(self, x1, x2):
        x1 = self.modelA(x1)  
        x1 = x1.view(x1.size(0), -1)
        x2 = self.modelB(x2)
        x2 = x2.view(x2.size(0), -1)
        x = torch.cat((x1, x2), dim=1)
        
        x = self.classifier(F.relu(x))
        return x

The output features will be concatenated in the torch.cat call.
Would that work for your “fusion” or are you thinking about another method?
Could you explain your concerns about the DataLoader?
How are you defining both Datasets currently and how would you like to sample each image?
If you want to use the same index to get the samples from both Datasets, I think the easiest approach would be to wrap these Datasets in another one and just index them:
class MyDataset(Dataset):
    def __init__(self, ds1, ds2):
        self.ds1 = ds1
        self.ds2 = ds2
        
    def __getitem__(self, index):
        x1, y1 = self.ds1[index]
        x2, y2 = self,ds2[index]
        return x1, y1, x2 ,y2
    
    def __len__(self):
        return len(self.ds1) # assume both datasets have same length