Elaborating : Create two torch.utils.Dataset classes of the two different data you have. Then create a third dataset class that has it’s element instances of those 2 classes. The __getitem__ method of this third fusion dataset class would call the 2 datasets with probability 4:1.
Or you could do something like :
def __init__(torch.utils.data.Dataset) : 
    self.data1 = #call first instance
    self.data2 = #call second instance
    self.size1 = 80
    self.size2 = 20

def __getitem__(self,index):
    if (index<self.size1):
        return self.data1[index]
    else: 
        return self.data2[index-self.size1]

That’s just a rough outline, you can add more elements to the class