I try to improve code again, it seems work now.
I have one more question, when I do memory check after "output =  model(input)"
it will also cost about 2G memory, is it temp feature map memory cost, should I be freed immediately when computing done?