Solved by ptrblck in post #4 
              
              
                You are most likely just measuring the kernel launch times in your second code snippet. 
To properly time a segment, you would have to synchronize before starting and stopping the timer. 
E.g. this code should show a high duration in the actual forward pass: 
while i < 10: 
    with torch.no_grad():…