Solved by ptrblck in post #2 
              
              
                Linear layers are initialized with 
stdv = 1. / math.sqrt(self.weight.size(1))
self.weight.data.uniform_(-stdv, stdv)
if self.bias is not None:
    self.bias.data.uniform_(-stdv, stdv)

See also here.