Solved by MariosOreo in post #4 
              
              
                Hello, 
Because the weight shape of new_last_layer wil mismatch the weight shape of old_last_layer, I think you should do some manaully, padding the old_last_layer weight to the same shape of new_last_layer weight with random values, zeros or ones, and then load_state_dict normally. 
I wrote a simpl…