I ran a few more quick what-if experiments to satisfy my curiosity.
I can achieve the highest GPU utilization on the MNIST demo, and see some expected performance scaling with higher batch size, with num_workers=0 and pin_memory=True … this is a little bit higher than num_workers=0 and pin_memory=False.
Setting pin_memory=True with any number of worker processes > 0 is an absolute disaster that pins all cores at 100%.