I was wondering what the reason is that weights are initialized the way they are? For torch.nn.Linear the weights are initialized with samples from Uniform(-sqrt(k), sqrt(k)) with k = 1/no_features. Linear — PyTorch 1.7.0 documentation 5
In the source code “kaiming_uniform” appears. However, to my understanding Kaiming He at al. [1] suggest sampling from a normal distribution. If I take the condition they state that 0.5 * no_features * VAR[w_l] = 1 and plug in the variance for the uniform distribution which should be (b - a)^2 / 12, with a= - sqrt(k) and b=sqrt(k), this does not add up either.
So what is the “Kaiming Uniform” and/or what is the reason that the weights are "initialized from  initialized from U}(-sqrt(k)), sqrt(k))?
Thanks for any help
[1] https://arxiv.org/pdf/1502.01852.pdf