Thanks a lot. It works after I modify the code to your suggestion.
However, correct me if I am wrong, I think pytorch  should come up with a clever way to for this gpu memeory control problem, for the good of this platform.