Solved by Carl in post #2 
              
              
                The simplest way to do it is to train certain layers while have the other layers act like the identity function. You can select what parameters you want to train when making your optimizer (https://pytorch.org/docs/stable/optim.html#per-parameter-options). To speed things up, you can avoid computing…