Hi,
I would guess your problem is the same as https://discuss.pytorch.org/t/checking-tensor-is-all-0s-tensor-sum-data-0-0-extremely-slow/16613.
Set the CUDA_LAUNCH_BLOCKING=1 environment variable before runnning your script in a profiler to avoid this behaviour