Deep Learning, Activation Functions, Machine Learning.
both Sigmoid and tanh activation function is “Saturation”. Large values translate to 1 & small ones to either 0 or -1. The functions are only sensitive in the region around mid point(0.5 0r 0). This makes weight adaptation very difficult. Due to the vanishing gradient problem attached with