You could consider skipping the file in the data loader and using a custom collate function to remove the missing values.
def my_collate(batch):
    "Puts each data field into a tensor with outer dimension batch size"
    batch = list(filter(lambda x:x is not None, batch))
    return default_collate(batch)
train_loader = torch.utils.data.DataLoader(
        train_dataset, batch_size=args.batch_size, shuffle=True,
        num_workers=args.num_workers, pin_memory=True, sampler=train_sampler,collate_fn=my_collate)

If you have a lot of missing images this might cause your batch size to fluctuate too much. In which case preprocessing the images offline and training with images that are in-tact will resolve your issue.