Sparse activation: For example, in a randomly initialized network, only about 50% of hidden units are activated (having a non-zero output). Better gradient propagation: Fewer vanishing gradient problems compared to sigmoidal activation functions that saturate in both directions.[1] Efficient computation: Only comparison, addition and multiplication. Scale-invariant: max ( 0 , a x ) = a max ( 0 , x ) for a ≥ 0 {\displaystyle \max(0,ax)=a\max(0,x){\text{ for }}a\geq 0} .