But the last two samples should be invalid, right?
If so, we would need the invalid indices calculated from your dataset.
If not, just remove the invalid index filters in my sampler:
paired_indices = torch.stack(
   [paired_indices[i] for i in range(len(paired_indices)) 
        if not i in invalid_idx])

and pass the sampler to your DataLoader:
loader = DataLoader(
    dataset,
    batch_size=1,
    sampler=sampler
)