Finally, I found the reason! Due to the limitation of the input size of ResNet, there was a miss-match between CNN layer and FC layer. When I change the resize value of transforms, it works well.
Thank you.