@liyz15, thanks for replying 
I get it now, that the error is because of the mismatch of the parameters
but i  can not remove the filters, since i am freezing some layers of the model.
What if after freezing the layers, i define a new optimizer ?
Would it be any different from loading the state_dict of the previously saved optimizer ?