Answers.Body: <p>They are <strong>not</strong> all interchangeable. However, all these expressions are related to each other and to the concept of optimization. Some of them are synonymous, but keep in mind that these terms may not be used consistently in the literature.</p>
  <p>In machine learning, a <strong>loss function</strong> is a function that computes the loss/error/cost, given a supervisory signal and the prediction of the model, although this expression might be used also in the context of unsupervised learning. The terms <strong>loss function, cost function</strong> or <strong>error function</strong> are often used interchangeably <a href="https://www.deeplearningbook.org/contents/numerical.html" rel="nofollow noreferrer">[1]</a>, <a href="https://hastie.su.domains/Papers/ESLII.pdf" rel="nofollow noreferrer">[2]</a>, <a href="http://noiselab.ucsd.edu/ECE228/Murphy_Machine_Learning.pdf" rel="nofollow noreferrer">[3]</a>. For example, you might prefer to use the expression <em>error function</em> if you are using the <em>mean squared error</em> (because it contains the term <em>error</em>), otherwise, you might just use any of the other two terms.</p>
  <p>In genetic algorithms, the <a href="https://ai.stackexchange.com/q/25872/2444"><strong>fitness function</strong></a> is any function that assesses the quality of an individual/solution <a href="https://cs.gmu.edu/%7Esean/book/metaheuristics/Essentials.pdf" rel="nofollow noreferrer">[4]</a>, <a href="http://www0.cs.ucl.ac.uk/staff/W.Langdon/ftp/papers/poli08_fieldguide.pdf" rel="nofollow noreferrer">[5]</a>, <a href="https://warin.ca/ressources/books/2015_Book_IntroductionToEvolutionaryComp.pdf" rel="nofollow noreferrer">[6]</a>, <a href="http://www.shahed.ac.ir/stabaii/Files/CompIntelligenceBook.pdf" rel="nofollow noreferrer">[7]</a>. If you are solving a supervised learning problem with genetic algorithms, it can be a synonym for error function <a href="https://dl.acm.org/doi/pdf/10.1145/3071178.3071330" rel="nofollow noreferrer">[8]</a>. If you are solving a reinforcement learning problem with genetic algorithms, it can also be a synonym for <a href="https://ai.stackexchange.com/q/25872/2444"><strong>reward function</strong></a> <a href="https://arxiv.org/pdf/1703.03864.pdf" rel="nofollow noreferrer">[9]</a>.</p>
  <p>In mathematical optimization, the <strong>objective function</strong> is the function that you want to optimize, either minimize or maximize. It's called the <em>objective function</em> because the objective of the optimization problem is to optimize it. So, this term can refer to an error function, fitness function, or any other function that you want to optimize. <a href="https://web.stanford.edu/%7Eboyd/cvxbook/bv_cvxbook.pdf" rel="nofollow noreferrer">[10]</a> states that the objective function is a utility function (<a href="https://web.stanford.edu/%7Eboyd/cvxbook/bv_cvxbook.pdf#page=24" rel="nofollow noreferrer">here</a>).</p>
  <p>A <a href="https://www.investopedia.com/ask/answers/072915/what-utility-function-and-how-it-calculated.asp" rel="nofollow noreferrer"><strong>utility function</strong></a> is usually the opposite or negative of an error function, in the sense that it measures a positive aspect. So, you want to maximize the utility function, but you want to minimize the error function. This term is more common in economics, but, sometimes, it is also used in AI <a href="https://sciendo.com/pdf/10.2478/v10229-011-0013-5" rel="nofollow noreferrer">[11]</a>.</p>
  <p>The term <strong>criterion function</strong> is not very common, at least, in machine learning. It could refer to the function that is used to stop an algorithm. For example, if you are executing a computationally expensive procedure, a stopping criterion might be time. So, in this case, your criterion function might return true after a certain number of seconds have passed. However, <a href="https://www.deeplearningbook.org/contents/numerical.html" rel="nofollow noreferrer">[1]</a> uses it as a synonym for the objective function.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-29T11:40:39.510Z
Answers.LastActivityDate: 2021-12-09T15:13:49.647Z
Answers.LastEditDate: 2021-12-09T15:13:49.647Z
Answers.UserDisplayName: nbro
Body: <p>I find the terms cost, loss, error, fitness, utility, objective, criterion functions to be interchangeable, but any kind of minor difference explained is appreciated.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-08-06T10:08:23.593Z
Comments.Text: Related thread at Cross Validated: [Objective function, cost function, loss function: are they the same thing?](https://stats.stackexchange.com/questions/179026/objective-function-cost-function-loss-function-are-they-the-same-thing)
Comments.UserDisplayName: desertnaut
Tags: <deep-learning><convolutional-neural-networks><terminology><objective-functions><comparison>
Title: What are the major differences between cost, loss, error, fitness, utility, objective, criterion functions?
UserDisplayName: Stephen Philip
ThreadText: # What are the major differences between cost, loss, error, fitness, utility, objective, criterion functions?
  
  (Asked by: username_0 on 2019-07-29)
  
  I find the terms cost, loss, error, fitness, utility, objective, criterion functions to be interchangeable, but any kind of minor difference explained is appreciated.
  
  **Tags:** deep-learning, convolutional-neural-networks, terminology, objective-functions, comparison
  > **Comments:**
  > **username_1**: Related thread at Cross Validated: [Objective function, cost function, loss function: are they the same thing?](https://stats.stackexchange.com/questions/179026/objective-function-cost-function-loss-function-are-they-the-same-thing)  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2019-07-29. Score: 5 | Accepted answer
  
  They are **not** all interchangeable. However, all these expressions are related to each other and to the concept of optimization. Some of them are synonymous, but keep in mind that these terms may not be used consistently in the literature.
  
  In machine learning, a **loss function** is a function that computes the loss/error/cost, given a supervisory signal and the prediction of the model, although this expression might be used also in the context of unsupervised learning. The terms **loss function, cost function** or **error function** are often used interchangeably [[1]](https://www.deeplearningbook.org/contents/numerical.html), [[2]](https://hastie.su.domains/Papers/ESLII.pdf), [[3]](http://noiselab.ucsd.edu/ECE228/Murphy_Machine_Learning.pdf). For example, you might prefer to use the expression *error function* if you are using the *mean squared error* (because it contains the term *error*), otherwise, you might just use any of the other two terms.
  
  In genetic algorithms, the [**fitness function**](https://ai.stackexchange.com/q/25872/2444) is any function that assesses the quality of an individual/solution [[4]](https://cs.gmu.edu/%7Esean/book/metaheuristics/Essentials.pdf), [[5]](http://www0.cs.ucl.ac.uk/staff/W.Langdon/ftp/papers/poli08_fieldguide.pdf), [[6]](https://warin.ca/ressources/books/2015_Book_IntroductionToEvolutionaryComp.pdf), [[7]](http://www.shahed.ac.ir/stabaii/Files/CompIntelligenceBook.pdf). If you are solving a supervised learning problem with genetic algorithms, it can be a synonym for error function [[8]](https://dl.acm.org/doi/pdf/10.1145/3071178.3071330). If you are solving a reinforcement learning problem with genetic algorithms, it can also be a synonym for [**reward function**](https://ai.stackexchange.com/q/25872/2444) [[9]](https://arxiv.org/pdf/1703.03864.pdf).
  
  In mathematical optimization, the **objective function** is the function that you want to optimize, either minimize or maximize. It's called the *objective function* because the objective of the optimization problem is to optimize it. So, this term can refer to an error function, fitness function, or any other function that you want to optimize. [[10]](https://web.stanford.edu/%7Eboyd/cvxbook/bv_cvxbook.pdf) states that the objective function is a utility function ([here](https://web.stanford.edu/%7Eboyd/cvxbook/bv_cvxbook.pdf#page=24)).
  
  A [**utility function**](https://www.investopedia.com/ask/answers/072915/what-utility-function-and-how-it-calculated.asp) is usually the opposite or negative of an error function, in the sense that it measures a positive aspect. So, you want to maximize the utility function, but you want to minimize the error function. This term is more common in economics, but, sometimes, it is also used in AI [[11]](https://sciendo.com/pdf/10.2478/v10229-011-0013-5).
  
  The term **criterion function** is not very common, at least, in machine learning. It could refer to the function that is used to stop an algorithm. For example, if you are executing a computationally expensive procedure, a stopping criterion might be time. So, in this case, your criterion function might return true after a certain number of seconds have passed. However, [[1]](https://www.deeplearningbook.org/contents/numerical.html) uses it as a synonym for the objective function.
  
  
  ---
  
  