Deep Learning, Activation Functions, Machine Learning.
inactive. Computationally it is slightly more expensive than ReLU or Leaky ReLU but it trains faster. Also it has better generalization as it is has a mean values closer to zero. It solves both vanishing & exploding gradient problems. Since output of the ReLU and its versions are unbounded in