Solved by Ricocotam in post #4 
              
              
                Look for “autograd” and how to freeze layers. 
If you don’t want to use Linear layers, you pick ones already existing or look how to build custom ones