Body: <p>Per page 7 of <a href="http://web.mit.edu/6.S097/www/resources/L01.pdf" rel="nofollow noreferrer">this MIT lecture notes</a>, the original single-layer Perceptron uses 0-1 loss function.</p>
  <p><a href="https://en.wikipedia.org/wiki/Perceptron" rel="nofollow noreferrer">Wikipedia</a> uses</p>
  <p><span class="math-container">$${\displaystyle {\frac {1}{s}}\sum _{j=1}^{s}|d_{j}-y_{j}(t)|} \tag{1}$$</span></p>
  <p>to denote the error.</p>
  <p>Is the formula (1) the correct form of 0-1 loss function?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-06-26T10:27:00.007Z
Comments.Text: There is no loss function for this particular case since this algorithm iteratively updates each weight separately. The wiki says this formula is used to stop training if it is less than a certain threshold. Take a look at [this](https://ai.stackexchange.com/questions/28235/where-does-the-so-called-loss-loss-function-fit-into-the-idea-of-a-percept/28283#28283) and [this](https://ai.stackexchange.com/questions/28395/an-explanation-involving-the-sign-activation-its-affect-on-the-loss-function-a/28396#28396) answers for more details.
Comments.UserDisplayName: Aray Karjauv
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-06-26T11:07:44.663Z
Comments.Text: @ArayKarjauv Thank you. What does "this particular case" refer to? Both sources claim there **is** a loss function.
Comments.UserDisplayName: JJJohn
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-06-26T11:59:23.033Z
Comments.Text: This (particular) iterative learning algorithm presented in the provided articles implies a non-differentiable loss function (usually `sign`) that does not quantify the predictions, but only says whether they were correct or not, so you cannot optimize it using gradient descent. Therefore, it is not a loss function in the usual sense. There is a loss function if your activation is differentiable. For instance, sigmoid. Then, the loss will be binary cross-entropy.
Comments.UserDisplayName: Aray Karjauv
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-06-26T20:03:52.803Z
Comments.Text: I've created a chat room about perceptrons. Feel free to join it: https://chat.stackexchange.com/rooms/126900/perceptron-learning-algorithm
Comments.UserDisplayName: Aray Karjauv
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-06-29T15:52:24.893Z
Comments.Text: @ArayKarjauv I don't think that a "loss function" necessarily needs to be differentiable, although this is probably just a matter of convention. Anyway, if you have an answer to this question, feel free to write it below. Comments are meant to be temporary and to provide additional info rather than full answers.
Comments.UserDisplayName: nbro
Tags: <neural-networks><objective-functions><definitions><perceptron>
Title: Is the formula $\frac {1}{s}\sum _{j=1}^{s}|d_{j}-y_{j}(t)|$ the correct form of 0-1 loss function, in the context of Perceptron?
UserDisplayName: JJJohn
ThreadText: # Is the formula $\frac {1}{s}\sum _{j=1}^{s}|d_{j}-y_{j}(t)|$ the correct form of 0-1 loss function, in the context of Perceptron?
  
  (Asked by: username_0 on 2021-06-26)
  
  Per page 7 of [this MIT lecture notes](http://web.mit.edu/6.S097/www/resources/L01.pdf), the original single-layer Perceptron uses 0-1 loss function.
  
  [Wikipedia](https://en.wikipedia.org/wiki/Perceptron) uses
  
  $${\displaystyle {\frac {1}{s}}\sum \_{j=1}^{s}|d\_{j}-y\_{j}(t)|} \tag{1}$$
  
  to denote the error.
  
  Is the formula (1) the correct form of 0-1 loss function?
  
  **Tags:** neural-networks, objective-functions, definitions, perceptron
  > **Comments:**
  > **username_1**: There is no loss function for this particular case since this algorithm iteratively updates each weight separately. The wiki says this formula is used to stop training if it is less than a certain threshold. Take a look at [this](https://ai.stackexchange.com/questions/28235/where-does-the-so-called-loss-loss-function-fit-into-the-idea-of-a-percept/28283#28283) and [this](https://ai.stackexchange.com/questions/28395/an-explanation-involving-the-sign-activation-its-affect-on-the-loss-function-a/28396#28396) answers for more details.  
  > **username_0**: @ArayKarjauv Thank you. What does "this particular case" refer to? Both sources claim there **is** a loss function.  
  > **username_1**: This (particular) iterative learning algorithm presented in the provided articles implies a non-differentiable loss function (usually `sign`) that does not quantify the predictions, but only says whether they were correct or not, so you cannot optimize it using gradient descent. Therefore, it is not a loss function in the usual sense. There is a loss function if your activation is differentiable. For instance, sigmoid. Then, the loss will be binary cross-entropy.  
  > **username_1**: I've created a chat room about perceptrons. Feel free to join it: https://chat.stackexchange.com/rooms/126900/perceptron-learning-algorithm  
  > **username_2**: @ArayKarjauv I don't think that a "loss function" necessarily needs to be differentiable, although this is probably just a matter of convention. Anyway, if you have an answer to this question, feel free to write it below. Comments are meant to be temporary and to provide additional info rather than full answers.  
  
  ---
  
  ## 0 answers
  
  