Solved by ptrblck in post #21 
              
              
                Thanks for the notebook. 
The DownConv layer returns a tuple in this line of code, which doesn’t work in an nn.Sequential container and standard layers. 
If you want to accept the tuple in the next layer, you could e.g. write a custom layer, which unwraps the tuple internally.