Machine Learning, Loss Function, Optimisation, Lua, Deep Learning.
Rectified Linear Unit (ReLU), Tan Hyperbolic (tanh) and Identity function. Error and Loss Function: In most learning networks, error is calculated as the difference between the actual output and the predicted output. The function that is used to compute this error is known as Loss Function J(.).