the bottleneck is usually the available number of CPU cores (usually 4 to 20 cores are available).
Trying to parallelize over both N and T through N * T workers wont help.
You probably want to just load one sample (with TxHxWx3) per worker thread.