This worked pretty great thanks.
I managed to get parallel loading working using the standard dataloader as described in my initial question (turns out the error I was getting was just because my batch size was too high)
But that didn’t make my pipeline any faster until I tried your Queue idea.

COpying data to GPU went from 5 seconds (serial) to 0.0002 seconds (hidden cost by Queue pipeline

Thank you