Does using log of classifier outputs improve training performance? When we train models, often we use the log of outputs in $(0,1)$ in our cost functions. Does anyone have a source that explains if this improves training?
For example, supposed the cost is $f(\theta)=\sum_i{\delta_{y_i=1}\log\hat{y}_i}$.
I think I read somewhere that stretching $(0,1)$ to $(-\infty,0)$ helps learning.
Does anyone have a reference?

Remember that:
$\sum_i \delta_{y_i = 1}\log( y_i) = \sum_i \log( y_i^{\delta_{y_i = 1}}) = \log(\prod_i y_i^{\delta_{y_i = 1}})$
In short, if you don't take the log you are multiplying a lot of numbers between 0 and 1 with each other. If you do that, you will end up with very very very small numbers. So small, that computers will have trouble acurately working with them.
