No, there’s no easy way to do it. DataLoader isn’t meant to load from multiple splits, but from a single dataset. One possible solution would be to call .share_memory_() on the tensor that holds the images. This way, when you fork the workers, they should inherit it (unless something I don’t remember about stops them).
Can you show me how you instantiate both training and validation data loaders?