Thanks for your reply,
I tried using this before .time() like this
torch.cuda.synchronize()
t1 = time.time()
...some operations here...
torch.cuda.synchronize()
t2 = time.time()
print("Time consumption:", )

I add torch.cuda.synchronize() before every time.time() and I have got quite a different result from the situation using only time.time() without torch.cuda.synchronize() . I finally localized the time consuming part and found the code below actually takes over 0.4s+. From what you told me, could I assume that torch.cuda.synchronize() could help me get absolute time of these operations ?
If is, then my question became:
# inputs shape: [1, 128, 120, 120]
torch.max(inputs.view(b, -1), dim=1)[0]  # very time consuming ~0.4s +
torch.max(inputs) # very quick ~ 0.0003s

Why is there the difference ? I couldn’t see the mechanism behind it.  From what I see it shouldn’t be like this