Answers.Body: <p>I see why you might be confused. First, the logistic-loss or log-loss is technically called cross-entropy loss. This function is very simple:</p>
  <p><span class="math-container">$CE = -[y \log(p) + (1 - y) \log(1 - p)]$</span></p>
  <p>This tells basically if the predicted class <span class="math-container">$y$</span> was right <span class="math-container">$y=1$</span> then the loss is <span class="math-container">$CE=-\log(p)$</span>, if the predicted class was not the right one then the loss is <span class="math-container">$CE=-\log(1-p)$</span>.</p>
  <p>If we look at the function as a pure math concept we see that:</p>
  <p><span class="math-container">$CE = f(x) = - \log(x)$</span></p>
  <p>And as you point out, that function is minima-unbounded as its domain is <span class="math-container">$D(f(x)) = [0, +\infty]$</span>. You can check that in here:</p>
  <p><a href="https://i.sstatic.net/dWM0J.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/dWM0J.png" alt="enter image description here" /></a></p>
  <p>However the trick is that the inputs must be bounded, meaning, the inputs to the loss function must be in range <span class="math-container">$[0, 1]$</span>. This bounding is achieved by applying a sigmoid activation function as the final &quot;layer&quot; of the network. Then if the inputs to the loss function are bounded the function has a clear minima.</p>
  <p>Check how the function looks in reality from one of the most important papers in loss function in the AI world: <a href="https://arxiv.org/abs/1708.02002" rel="nofollow noreferrer">Focal Loss</a> (I really encourage you to read it as the first section explains in detail the cross-entropy loss). The blue curve is the one you are looking for.</p>
  <p><a href="https://i.sstatic.net/U0AbV.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/U0AbV.png" alt="enter image description here" /></a></p>
  <p>Finally, you might want to review your log-loss/CE function since it should have an asymptote for <span class="math-container">$f(x=0) = \infty$</span></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-02T11:24:35.600Z
Answers.LastActivityDate: 2020-08-02T11:24:35.600Z
Answers.UserDisplayName: JVGD
Body: <p>Normally, in practice, people use those loss functions with minima, e.g. <span class="math-container">$L_1$</span> mean absolute loss, <span class="math-container">$L_2$</span> mean squared error, etc. All those come with a minimum to optimize to.
  <a href="https://i.sstatic.net/RSccA.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/RSccA.png" alt="enter image description here" /></a></p>
  <p>However, there's another thing, logistic loss, I'm reading about, but don't get it why the logistic function could be used as a loss function, given that it has the so-called minimum at infinity, but that isn't a normal minimum. Logistic loss function (black curve):</p>
  <p><a href="https://i.sstatic.net/0pREW.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/0pREW.png" alt="enter image description here" /></a></p>
  <p>How can an optimizer minimize the logistic loss?</p>
  
Tags: <objective-functions><optimization><logistic-regression><neural-networks>
Title: How could logistic loss be used as loss function for an ANN?
UserDisplayName: Dan D.
ThreadText: # How could logistic loss be used as loss function for an ANN?
  
  (Asked by: username_0 on 2020-07-31)
  
  Normally, in practice, people use those loss functions with minima, e.g. $L\_1$ mean absolute loss, $L\_2$ mean squared error, etc. All those come with a minimum to optimize to.
  [![enter image description here](https://i.sstatic.net/RSccA.png)](https://i.sstatic.net/RSccA.png)
  
  However, there's another thing, logistic loss, I'm reading about, but don't get it why the logistic function could be used as a loss function, given that it has the so-called minimum at infinity, but that isn't a normal minimum. Logistic loss function (black curve):
  
  [![enter image description here](https://i.sstatic.net/0pREW.png)](https://i.sstatic.net/0pREW.png)
  
  How can an optimizer minimize the logistic loss?
  
  **Tags:** objective-functions, optimization, logistic-regression, neural-networks
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-08-02. Score: 2 | Accepted answer
  
  I see why you might be confused. First, the logistic-loss or log-loss is technically called cross-entropy loss. This function is very simple:
  
  $CE = -[y \log(p) + (1 - y) \log(1 - p)]$
  
  This tells basically if the predicted class $y$ was right $y=1$ then the loss is $CE=-\log(p)$, if the predicted class was not the right one then the loss is $CE=-\log(1-p)$.
  
  If we look at the function as a pure math concept we see that:
  
  $CE = f(x) = - \log(x)$
  
  And as you point out, that function is minima-unbounded as its domain is $D(f(x)) = [0, +\infty]$. You can check that in here:
  
  [![enter image description here](https://i.sstatic.net/dWM0J.png)](https://i.sstatic.net/dWM0J.png)
  
  However the trick is that the inputs must be bounded, meaning, the inputs to the loss function must be in range $[0, 1]$. This bounding is achieved by applying a sigmoid activation function as the final "layer" of the network. Then if the inputs to the loss function are bounded the function has a clear minima.
  
  Check how the function looks in reality from one of the most important papers in loss function in the AI world: [Focal Loss](https://arxiv.org/abs/1708.02002) (I really encourage you to read it as the first section explains in detail the cross-entropy loss). The blue curve is the one you are looking for.
  
  [![enter image description here](https://i.sstatic.net/U0AbV.png)](https://i.sstatic.net/U0AbV.png)
  
  Finally, you might want to review your log-loss/CE function since it should have an asymptote for $f(x=0) = \infty$
  
  
  ---
  
  