Answers.Body: <p>Sounds like it worked to me.</p>
  <p><a href="https://pytorch.org/docs/stable/generated/torch.nn.LogSoftmax.html" rel="nofollow noreferrer">nn.LogSoftmax</a> returns the log of the softmax (duh). The outputs from softmax add up to 1, and form a probability distribution.</p>
  <p>0 is the log of 1, meaning that class was predicted at a level of nearly 100%. And the other classes with large negative logs are a rounding error.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-10T17:15:28.527Z
Answers.LastActivityDate: 2021-09-10T17:15:28.527Z
Answers.UserDisplayName: Criticize SE actions means ban
Body: <p>I have trained a classification network with PyTorch lightning where my training step looks like below:</p>
  <pre><code>def training_step(self, batch, batch_idx):
      x, y = batch
      y_hat = self(x)
      loss = F.cross_entropy(y_hat, y)
      self.log(&quot;train_loss&quot;, loss, on_epoch=True)
      return
  
  </code></pre>
  <p>When I look at the output logits, almost all of them are very large negative numbers, with one that is usually 0. Is this normal or might be something wrong with my training?</p>
  <p>I am just using <code>nn.LogSoftmax()</code> on the outputs and taking the max to make my predictions, but my network is not doing so good when I am running on unseen data, and I want to make sure the problem is just me overfitting.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-04-28T18:40:08.647Z
Comments.Text: Kind of yes... It's log of softmax. Now softmax will always be < 1 and > 0. Apart from the top class, rest would be close to 0. So log of that will be large negative number.
Comments.UserDisplayName: user1953366
Tags: <deep-learning><classification><pytorch><softmax>
Title: Is it normal that the values of the LogSoftmax function are very large negative numbers?
UserDisplayName: pd109
ThreadText: # Is it normal that the values of the LogSoftmax function are very large negative numbers?
  
  (Asked by: username_0 on 2021-09-09)
  
  I have trained a classification network with PyTorch lightning where my training step looks like below:
  def training_step(self, batch, batch_idx):
      x, y = batch
      y_hat = self(x)
      loss = F.cross_entropy(y_hat, y)
      self.log("train_loss", loss, on_epoch=True)
      return
  
  
  When I look at the output logits, almost all of them are very large negative numbers, with one that is usually 0. Is this normal or might be something wrong with my training?
  I am just using nn.LogSoftmax() on the outputs and taking the max to make my predictions, but my network is not doing so good when I am running on unseen data, and I want to make sure the problem is just me overfitting.
  
  **Tags:** deep-learning, classification, pytorch, softmax
  > **Comments:**
  > **username_1**: Kind of yes... It's log of softmax. Now softmax will always be < 1 and > 0. Apart from the top class, rest would be close to 0. So log of that will be large negative number.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-09-10. Score: 1 | Accepted answer
  
  Sounds like it worked to me.
  
  [nn.LogSoftmax](https://pytorch.org/docs/stable/generated/torch.nn.LogSoftmax.html) returns the log of the softmax (duh). The outputs from softmax add up to 1, and form a probability distribution.
  
  0 is the log of 1, meaning that class was predicted at a level of nearly 100%. And the other classes with large negative logs are a rounding error.
  
  
  ---
  
  