This should be the solution to our issue: https://github.com/pytorch/vision/issues/1128 256
However, we will have to skip those images without annotations. So I have followed the solution given here: Ignore images without annotations 105
Good luck!