Solved by ptrblck in post #4 
              
              
                Using nn.Sequential will give you the same dict with layer names defined as ascending numbers. 
Try this as a model definition and run the loop with it: 
model_seq = nn.Sequential(
    nn.Linear(1, 5),
    nn.ReLU(),
    nn.Linear(5, 1),
    nn.LogSigmoid()
)

EDIT: Alternatively you could use an Or…