which is called the softplus[10][2] or SmoothReLU function.[11] The derivative of softplus is f ′ ( x ) = e x 1 + e x = 1 1 + e − x {\displaystyle f'(x)={\frac {e^{x}}{1+e^{x}}}={\frac {1}{1+e^{-x}}}} , the logistic function. The logistic sigmoid function is a smooth approximation of the derivative of the rectifier, the Heaviside step function.