Hello everyone
I am trying to initialize the layers of my network using kaiming_normal_ initializer. This method has an argument called mode with fan_in and fan_out options.
I read the docs and found out it depends on the number of filters or channels. For example, I have a Conv2d layer with size of  [64, 3, 4, 4]. When I calculate the values, I get fan_in=48 and fan_out=1024 which is a huge difference and result in std = gain/math.sqrt(fan) is like this:
fan_in enabled: std=0.2
fan_out enabled: std=0.04
And std is passed to tensor.normal_ method.
The question is I did not find any specific explanation about which mode should be chose in the paper I am trying to implement or other resources.
Best regards