Thanks for your reply @richard @SimonW!
Yes. During my model training time I test the convolution function time and found the calculation is actually as fast as the last two GPU test case shown.
Maybe there is another reason increases the computing time during backword. I will try to find it and show it sometimes later:smiley: thank you.