PyTorch tries to max out the GPU utilization for the executed operations, such that multiple script executions might not be able to run in parallel.
The CPU time might accumulate the GPU execution time, but I’m not completely sure, as I’m usually either profile operations in isolation (using torch.cuda.synchronize() and a timer) or with nvprof/nsys.