Data Science, Activation Functions, Neural Networks, Bias, Deep Learning.
problem called Dying ReLU. Leaky ReLU Function This function is expressed as It is a variant of the ReLU function. It has a small slope for all negative values (α). α is mostly 0.01. Parametric ReLU Function:- Here, the parameter is sent to a neural network and that network learns the optimal value