In case the class distribution (4 landscape samples + 1 face sample) is not a hard requirement, you could use a WeightedRandomSampler and specify the sample weights in such a way that the desired distribution would be used in the “average batch”.
However, if the distribution is fixed, your approach seems like the best one, i.e. create a custom sampler and yield the precomputed indices to the Dataset.