Solved by ptrblck in post #10 
              
              
                Sorry for the late reply. I’ve reproduced the timing issue, profiled your code using torch.utils.bottleneck, and it seems run with random weights just performs a lot more calls to layers/box_utils.py. 
It’s a guess as I’m not familiar with the code, but I think the random weights just might create a…