Deep Learning, Activation Functions, Machine Learning.
activated. 2. Computationally cheap. 3. Faster convergence (Almost 6 times faster than Sigmoid or tanh) 4. Unlike Sigmoid or tanh this doesn’t have vanishing gradient problem 5. Suffers with problem called “dying ReLU”. Dying ReLU: Once a neuron gets negative, its very unlikely that it will