Solved by ptrblck in post #2 
              
              
                You could load the state_dict before any manipulation of the first conv layer to make sure all other layers are properly restored. 
Once this is done, you could replace the first conv layer with a new one and reassign the parameters manually. This code snippet give you an example: 
ref = nn.Conv2d(3…