You could probably create a custom sampler and try to pass more arguments to __getitem__.
However, what changes are you applying to your custom Dataset?
Note that each worker will work on a copy of the Dataset, so manipulating it is not trivial and you might need to use shared arrays/dicts etc.