Solved by blueeagle in post #2 
              
              
                I think I found the solution by myself. 
For everyone struggling with the same problem: 
You can use ModuleList. I my example, I can just append each encoder and the classifier to the ModuleList. Using this class, my “Main-Model” is aware of its submodels and for example the number of parameters is …