One approach would be to create separate DataLoaders with batch_size=5 (small dataset) and =27 (large dataset), respectively.
In the outer loop you would iterate the large dataset to make sure you are sampling all data points.
You could create an iterator for the small dataset via small_iter = iter(small_loader) before entering the loop and sample the batch from the small dataset.
Once you have both batches, you could concatenate them via data = torch.cat((small, large), dim=0) to create your final batch.
I think the WeightedRandomSampler wouldn’t strictly work, as you need to sample all samples from the large dataset.