Solved by 111526 in post #3 
              
              
                Finally, I found the reason! Due to the limitation of the input size of ResNet, there was a miss-match between CNN layer and FC layer. When I change the resize value of transforms, it works well. 
Thank you.