Solved by ptrblck in post #2 
              
              
                The computation graph is constructed in each forward pass, which allows you to create your model dynamically and use plain python control-flow operators like for loops etc.