I was wondering, how are biases initialized in pytorch for the linear layer? I inspected it out of curiosity and it seems randn(0,1) would be my guess:
>>> l = torch.nn.Linear(3,2)
>>> l.bias
Parameter containing:
0.2137
0.0904
[torch.FloatTensor of size 2]