Relu, Neural Networks, Deep Learning, Data Science, Machine Learning.
just a linear regression. One of the most commonly used activation functions is Rectified Linear Unit (ReLU), which has been theoretically shown to enable NNs to approximate a wide range of continuous functions, making them powerful function approximators. In this post, we study in particular the