Solved by tom in post #2 
              
              
                My understanding (the last update is here) is that this is mainly from casting the init as it has always been done to the kaiming_uniform_ initialization scheme. I would not read it as a recommendation for that particular activation function. 
As such, for me there are two takeways: 

This is a sche…