Hi,
I’m adapting the DCGAN models from the PyTorch examples and had a problem saving and loading the parameters.
Specifically, when I saved the parameters and then read them back in at a later stage, I would get a different output for a consistent input.
I debugged and found what I believed to be the problem, however I’m not sure of the cause. I am using a loop to define hidden layers of the MLP (I’m not using the conv nets):
self.nhl = 4
self.nh = [2, 2, 2, 2]
self.fc = []
for i in range(self.nhl-1):
    self.fc.append(nn.Linear(self.nh[i], self.nh[i+1]))

which gave the following output on repeated executions of the program (with a model saved earlier):
 jordan [ src ] $ python main.py 

First layer --> 0.0268119424582 0.0
hidden layer --> 0.0
hidden layer --> 0.558199584484
hidden layer --> 0.203323662281
output --> 0.305809795856 -0.139149516821 0.538217186928

 jordan [ src ] $ python main.py 
First layer --> 0.0268119424582 0.0
hidden layer --> 0.0
hidden layer --> 0.45568972826
hidden layer --> 0.0742047131062
output --> 0.425929844379 -0.366338938475 0.542093634605
 
jordan [ src ] $ python main.py 
First layer --> 0.0268119424582 0.0
hidden layer --> 0.0
hidden layer --> 0.0
hidden layer --> 0.460717827082
output --> 0.345742940903 -0.0932856351137 0.694804787636

Note that the outputs at each of the layers changes on each execution (I confirmed the parameters loaded are all consistent on each iteration).
Forward is implemented as:
    x = F.relu(self.input(_input))

    print 'First layer -->', x.data[0][0], x.data[0][1]
    for i in range(self.nhl-1):
        x = F.relu(self.fc[i]((x)))
        print 'hidden layer -->', x.data[0][0]

    output = self.output(x)

If I unroll the loops then it produces consistent output. Is there a bug in my loop or is there a reason you can’t use loops to define hidden layers?