Hi Asm!



 ASMIftekhar:

I have a few linear layers which I was not using, so I commented them out and surprisingly my Average Precision for first epoch moves from 34 to 33.
      self.lin_unused=nn.Sequential(nn.Linear(1024, 1024),nn.ReLU(),)
      self.lin_used=nn.Sequential(nn.Linear(1024, 512),nn.ReLU(),)



By default, nn.Linear initializes itself with some pseudorandom
numbers.  Thus, lin_unused consumes some pseudorandom
numbers and the pseudorandom numbers that lin_used uses
for its initialization will  differ, depending on whether lin_unused
is there or not.
You can check this in a number of ways:
First, print out some of the weights in lin_used and see that
they differ in the two cases.
Second, call torch.manual_seed with a fixed, known seed
immediately before constructing lin_used and check that
your results are the same whether or not you comment out
lin_unused.
Third, run your whole script a bunch of times (constructing and
running your model from scratch each time), passing a different
seed to  torch.manual_seed at the beginning of your script
each time, and verify that the results of your two versions
(with and without lin_unused) are statistically the same,
even though individual runs will differ from one another in
detail, due to statistical variation.
Best.
K. Frank