Deep Learning, Activation Functions, Machine Learning.
non-linear activation functions (sigmoid & tanh), they are not preferred in networks with many layers. We use Linear approximations. To be precise Rectified Linear Units. ReLU (Rectified Linear Unit) 1. Linear for all positive values and zero for all negative values. Hence this is sparsely