Thanks, I will try that…
For the code, I simply change the model first and then push it to cuda device. Compared to the original model, the modified version cause cuda memory went from 7GB to 15GB.
And also if I add a dropout layer after fc6 layer by using model.add_module(Dropout), is completely fine