Answers.Body: <p>The log-likelihood function for the training set (in general, not for deep learning in particular) will depend on your choice of loss function.</p>
  
  <p>I'm guessing you're using something like a quadratic <a href="https://en.wikipedia.org/wiki/Loss_functions_for_classification#Square_loss" rel="nofollow noreferrer">loss function</a> for a binary classification problem, since this is a common approach. In that case, the log-likelihood is the sum of logs of the squared differences between the target label and the softmax value produced by your model. If you want to compute the log likelihood under a particular set of parameters (say, those that minimize validation error), then you just use those parameters in the model when generating softmax values.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-08-09T06:16:56.830Z
Answers.Text: I too thought on the similar lines. I thought of using `negative loss likelihood loss` as given [here](https://pytorch.org/docs/stable/nn.html#nllloss). But this gives us negative log-likelihood loss whereas we are interested in calculating only log-likelihood. Could you please clarify my doubt?
Answers.UserDisplayName: Akhilesh Pandey
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-08-09T13:40:01.773Z
Answers.Text: I suspect that multiplying by -1 will give you the log likelihood then. Or you could minimize instead of maximizing.
Answers.UserDisplayName: John Doucette
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-08-08T07:50:41.017Z
Answers.LastActivityDate: 2018-08-08T07:50:41.017Z
Answers.UserDisplayName: John Doucette
Body: <p>I am building a supervised learning model and I wish to compute the log-likelihood for the training set at the point of the minimum validation error. </p>
  
  <p>Initially, I was computing the sum of all the probabilities with maximum value obtained after applying softmax for each example in the training set at the point of minimum validation error but that doesn't look correct. </p>
  
  <p>What is the correct formula for the log-likelihood?</p>
  
Tags: <machine-learning><training><optimization>
Title: How do I compute log-likelihood for training set in supervised learning?
UserDisplayName: Akhilesh Pandey
ThreadText: # How do I compute log-likelihood for training set in supervised learning?
  
  (Asked by: username_0 on 2018-08-08)
  
  I am building a supervised learning model and I wish to compute the log-likelihood for the training set at the point of the minimum validation error.
  
  Initially, I was computing the sum of all the probabilities with maximum value obtained after applying softmax for each example in the training set at the point of minimum validation error but that doesn't look correct.
  
  What is the correct formula for the log-likelihood?
  
  **Tags:** machine-learning, training, optimization
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-08-08. Score: 1 | Accepted answer
  
  The log-likelihood function for the training set (in general, not for deep learning in particular) will depend on your choice of loss function.
  
  I'm guessing you're using something like a quadratic [loss function](https://en.wikipedia.org/wiki/Loss_functions_for_classification#Square_loss) for a binary classification problem, since this is a common approach. In that case, the log-likelihood is the sum of logs of the squared differences between the target label and the softmax value produced by your model. If you want to compute the log likelihood under a particular set of parameters (say, those that minimize validation error), then you just use those parameters in the model when generating softmax values.
  
  > **Comments:**
  > **username_0**: I too thought on the similar lines. I thought of using `negative loss likelihood loss` as given [here](https://pytorch.org/docs/stable/nn.html#nllloss). But this gives us negative log-likelihood loss whereas we are interested in calculating only log-likelihood. Could you please clarify my doubt?  
  > **username_1**: I suspect that multiplying by -1 will give you the log likelihood then. Or you could minimize instead of maximizing.  
  
  ---
  
  