Solved by ptrblck in post #2 
              
              
                Yes, the assignment of the new module will properly register the new layer with all parameters and is the recommended way to e.g. replace the last linear layer in a fine-tuning use case (assuming the number of classes changed and you would like to retrain this layer). The parameters are also shown i…