I have 12 cores.
With num_workers=0, the utilization is 5-6 cores@100% and the rest @10-15%
With num_workers=1, the utilization is @50-60% on all cores.
With num_workers=2, the utilization is @60-70% on all cores.
I am measuring time by profiling with NVIDIA nsight-systems

  
      
      NVIDIA Developer – 12 Mar 18
  
  
    

NVIDIA Nsight Systems 3

NVIDIA® Nsight™ Systems is a system-wide performance analysis tool designed to visualize an application’s algorithms, help you identify the largest opportunities to optimize, and tune to scale efficiently across any quantity or size of CPUs and GPUs;...