This is expected, since the cpu() call is blocking and waiting for all asynchronous CUDA operations to finish. If you don’t synchronize manually, the cpu() call will add the synchronization and thus accumulate the CUDA kernel runtime.
Since you are now synchronizing manually, the runtime is spent in the kernel and the cpu() call executes just the device2host copy.



 Si_laopi:

is there any way to reduce such time? Like reduce image number of the input tensor?


Yes, you could reduce the batch size or the spatial size of the input images.
However, note that while reducing the workload the CUDA operations might reduce the wall-clock time, but the overall performance might be reduces, if you calculate the time/inputs.