Answers.Body: <p>You have several flaws in your code that will lead to unexpected behavior. I identified the following flaws you need to address. Once you have fixed them, you should output your updated weights during each step of training to see what the learning algorithm actually does and if it is going in the right direction. I will not address pure style aspects (like using a simpler case-statement instead of a series of ifs and stuff like that) and focus on the real errors.</p>
  
  <p><strong>Skipping training case 1</strong></p>
  
  <p>Once you are in training case 4, you set j to 1, expecting to use training case 1 next. But later in your code you increase j by one (j++) and go directly to training case 2, skipping the first one. This means you only run through training case 1 during your first pass of the loop.</p>
  
  <p><strong>Assigning j a value instead of comparing it</strong></p>
  
  <blockquote>
    <p>if(j=1)</p>
  </blockquote>
  
  <p>The if statement will always be true, because you do not compare j to 1 but you set j to 1. You basically assign a value and test if the value is true. Correct would be:</p>
  
  <blockquote>
    <p>if(j==1)</p>
  </blockquote>
  
  <p><strong>Ignoring bias during test</strong></p>
  
  <p>During the test steps after training you forget to add the bias that you used during training:</p>
  
  <blockquote>
    <p>output = weight1*input1 + weight2*input2</p>
  </blockquote>
  
  <p>should actually be:</p>
  
  <blockquote>
    <p>output = weight1*input1 + weight2*input2 + 2</p>
  </blockquote>
  
  <p>Otherwise your perceptron behaves differently during training and testing.</p>
  
  <p><strong>Perceptron output must be 1 or 0</strong></p>
  
  <p>Those were all implementation issues. This point actually seems to come from a misunderstanding of perceptrons. A real perceptron can either fire or not, meaning it outputs either 1 or 0, nothing in between. You calculate your output the following way and use the output to compare to the expected output:</p>
  
  <blockquote>
    <p>output = weight1*training1 + weight2*training2 + 2;</p>
    
    <p>if(output != expectedoutput )</p>
  </blockquote>
  
  <p>Your output here is a float value. Only in edge cases it will result in 1 or 0. What you actually want to do for a real perceptron is:</p>
  
  <blockquote>
    <p>if (weight1*training1 + weight2*training2 + 2 >= 0) {
     output = 1
    } else {
     output = 0
    }</p>
    
    <p>if(output != expectedoutput)</p>
  </blockquote>
  
  <p>After you fixed those errors and studied the output of your learning algorithms, you should be able to get the perceptron to work. If you have any questions, please leave a comment and I will try to help.</p>
  
  <hr>
  
  <p>Edit:</p>
  
  <p>I don't have a C compiler here, so I quickly implemented your solution with my suggested fixes in Python 3.6 and the results are very close to the correct solution. The final weights obviously have to be -1 and -1 for an AND Gate with bias 2. The learning process gets very close but never reaches -1 and -1 exactly, which leads to the wrong output in the end. This is a very good showcase to illustrate why you prefer a <strong>sigmoid neuron</strong> instead of a <strong>perceptron</strong> in modern neural networks. <a href="https://stats.stackexchange.com/questions/167478/what-is-the-difference-between-sigmoid-neurons-and-stochastic-binary-neurons">Here</a> is a relevant question from cross-validated concerning the difference of those two types of neurons.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-28T16:47:49.637Z
Answers.Text: I did the changes as suggested and indeed the learning process tends to 1. However i have a few questions.
    
  1) why do the weights tend to -1 and -1? if i input 1 1, the output is -1*1 + -1*1 +2 =0. But the answer should be 1 according to AND truth table. It seems that the output is still not correct?
  
  2) could someone explain or provide a link to the importance of learning rate here? i realize that the weights tend more to -1 if i use a small rate like 0.00015. and it gets less accurate when i use a higher learning rate. How do i determine the correct learning rate to use? 
  
  Thank you!
Answers.UserDisplayName: Sheep
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-28T21:43:19.083Z
Answers.Text: It tends towards 0 because 0 (or above) is the threshold for the perceptron to fire. Firing means "output 1" and not firing means "output 0". Using 0 is kind of arbitrary here, but that's what's usually used for perceptrons so I followed this convention. The learning rate just tells you how fast you are learning in each steps. A slower learning rate will require more training cycles but might get you closer to the "perfect" values in the end. There are mechanisms to determine a good constant or even dynamic learning rate, but in the beginning it is mostly trial and error. Hope that helps!
Answers.UserDisplayName: Demento
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-27T06:41:06.693Z
Answers.LastActivityDate: 2017-08-27T07:11:44.717Z
Answers.LastEditDate: 2017-08-27T07:11:44.717Z
Answers.UserDisplayName: Demento
Body: <p>I was trying to code a single layer perceptron to understand binary AND:</p>
  
  <p>1 1 1<br>
  0 1 0<br>
  1 0 0<br>
  0 0 0  </p>
  
  <p>I made up this code </p>
  
  <pre><code>#include &lt;stdio.h&gt;
  #include &lt;stdlib.h&gt;
  #include &lt;math.h&gt;
  int main()
  {
  int input1, input2;
  float weight1 = 0.3, weight2 = 0.4;
  int output;
  int training1, training2, expectedoutput;
  int i;
  int j=1;
  
  //TRAINING
  for(i=0; i&lt;10000;i++)
  {   
  
  
      if(j=1)
      {
          training1 = 0;
          training2 = 1;
          expectedoutput = 0;
      }
      if(j=2)
      {
          training1 = 1;
          training2 = 0;
          expectedoutput = 0;
      }
      if(j=3)
      {
          training1 = 0;
          training2 = 0;
          expectedoutput = 0;
      }
      if(j=4)
      {
          training1 = 1;
          training2 = 1;
          expectedoutput = 1;
          j=1;
      }
      output = weight1*training1 + weight2*training2 + 2;
  
      if(output != expectedoutput )
      {
          weight1 = weight1 + 0.156 * training1 * (expectedoutput - output);
          weight2 = weight2 + 0.156 * training2 * (expectedoutput - output);
      }
      j++;
  }
  
  printf("training done\n");
  printf("weight1 = %f" "weight2 = %f\n",weight1,weight2);
  
  //TESTING THE PERCEPTRON
  for(i=0; i&lt;5 ; i++)
  {
  scanf ("%d%d", &amp;input1, &amp;input2 );
  output = weight1*input1 + weight2*input2;
  printf("\n%d\n", output);
  }
  return 1;    
  }
  </code></pre>
  
  <p>its supposed to input the 4 cases repeatedly and with a learning rate of 0.156 (which i set randomly) and i used the threshold as a weight of 2. </p>
  
  <p>However after the training the perceptron still doesnt give the expected output. Is my understanding of perceptron rule wrong? Please help thank you!</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-08-27T20:30:39.203Z
Comments.Text: For debugging the actual code, you probably want to try a different Stack such as Code Review or Cross Validated.  The part of this question related to your understanding of the perceptron rule is definitely on topic here on the general AI forum.  Welcome to AI!
Comments.UserDisplayName: DukeZhou
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-02-13T17:40:13.760Z
Comments.Text: This question will be probably valid for this site if you present the algorithm (lets say, a short pseudocode) instead of the C source.
Comments.UserDisplayName: pasaba por aqui
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-02-13T17:41:10.203Z
Comments.Text: (Hint: do not fix threshold = 2, learn it).
Comments.UserDisplayName: pasaba por aqui
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-02-13T17:42:50.037Z
Comments.Text: Improvement: as x and y = y and x, you need only one weight.
Comments.UserDisplayName: pasaba por aqui
Tags: <neural-networks><machine-learning><artificial-neuron>
Title: debugging perceptron for digital AND circuit
UserDisplayName: Sheep
ThreadText: # debugging perceptron for digital AND circuit
  
  (Asked by: username_0 on 2017-08-26)
  
  I was trying to code a single layer perceptron to understand binary AND:
  
  1 1 1  
  0 1 0  
  1 0 0  
  0 0 0
  
  I made up this code
  
  ```
  #include <stdio.h>
  #include <stdlib.h>
  #include <math.h>
  int main()
  {
  int input1, input2;
  float weight1 = 0.3, weight2 = 0.4;
  int output;
  int training1, training2, expectedoutput;
  int i;
  int j=1;
  
  //TRAINING
  for(i=0; i<10000;i++)
  {   
  
      if(j=1)
      {
          training1 = 0;
          training2 = 1;
          expectedoutput = 0;
      }
      if(j=2)
      {
          training1 = 1;
          training2 = 0;
          expectedoutput = 0;
      }
      if(j=3)
      {
          training1 = 0;
          training2 = 0;
          expectedoutput = 0;
      }
      if(j=4)
      {
          training1 = 1;
          training2 = 1;
          expectedoutput = 1;
          j=1;
      }
      output = weight1*training1 + weight2*training2 + 2;
  
      if(output != expectedoutput )
      {
          weight1 = weight1 + 0.156 * training1 * (expectedoutput - output);
          weight2 = weight2 + 0.156 * training2 * (expectedoutput - output);
      }
      j++;
  }
  
  printf("training done\n");
  printf("weight1 = %f" "weight2 = %f\n",weight1,weight2);
  
  //TESTING THE PERCEPTRON
  for(i=0; i<5 ; i++)
  {
  scanf ("%d%d", &input1, &input2 );
  output = weight1*input1 + weight2*input2;
  printf("\n%d\n", output);
  }
  return 1;    
  }
  
  ```
  
  its supposed to input the 4 cases repeatedly and with a learning rate of 0.156 (which i set randomly) and i used the threshold as a weight of 2.
  
  However after the training the perceptron still doesnt give the expected output. Is my understanding of perceptron rule wrong? Please help thank you!
  
  **Tags:** neural-networks, machine-learning, artificial-neuron
  
  > **Comments:**
  > **username_1**: For debugging the actual code, you probably want to try a different Stack such as Code Review or Cross Validated.  The part of this question related to your understanding of the perceptron rule is definitely on topic here on the general AI forum.  Welcome to AI!  
  > **username_2**: This question will be probably valid for this site if you present the algorithm (lets say, a short pseudocode) instead of the C source.  
  > **username_2**: (Hint: do not fix threshold = 2, learn it).  
  > **username_2**: Improvement: as x and y = y and x, you need only one weight.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2017-08-27. Score: 0 | Accepted answer
  
  You have several flaws in your code that will lead to unexpected behavior. I identified the following flaws you need to address. Once you have fixed them, you should output your updated weights during each step of training to see what the learning algorithm actually does and if it is going in the right direction. I will not address pure style aspects (like using a simpler case-statement instead of a series of ifs and stuff like that) and focus on the real errors.
  
  **Skipping training case 1**
  
  Once you are in training case 4, you set j to 1, expecting to use training case 1 next. But later in your code you increase j by one (j++) and go directly to training case 2, skipping the first one. This means you only run through training case 1 during your first pass of the loop.
  
  **Assigning j a value instead of comparing it**
  
  > if(j=1)
  
  The if statement will always be true, because you do not compare j to 1 but you set j to 1. You basically assign a value and test if the value is true. Correct would be:
  
  > if(j==1)
  
  **Ignoring bias during test**
  
  During the test steps after training you forget to add the bias that you used during training:
  
  > output = weight1\*input1 + weight2\*input2
  
  should actually be:
  
  > output = weight1\*input1 + weight2\*input2 + 2
  
  Otherwise your perceptron behaves differently during training and testing.
  
  **Perceptron output must be 1 or 0**
  
  Those were all implementation issues. This point actually seems to come from a misunderstanding of perceptrons. A real perceptron can either fire or not, meaning it outputs either 1 or 0, nothing in between. You calculate your output the following way and use the output to compare to the expected output:
  
  > output = weight1\*training1 + weight2\*training2 + 2;
  >
  > if(output != expectedoutput )
  
  Your output here is a float value. Only in edge cases it will result in 1 or 0. What you actually want to do for a real perceptron is:
  
  > if (weight1\*training1 + weight2\*training2 + 2 >= 0) {
  > output = 1
  > } else {
  > output = 0
  > }
  >
  > if(output != expectedoutput)
  
  After you fixed those errors and studied the output of your learning algorithms, you should be able to get the perceptron to work. If you have any questions, please leave a comment and I will try to help.
  
  ---
  
  Edit:
  
  I don't have a C compiler here, so I quickly implemented your solution with my suggested fixes in Python 3.6 and the results are very close to the correct solution. The final weights obviously have to be -1 and -1 for an AND Gate with bias 2. The learning process gets very close but never reaches -1 and -1 exactly, which leads to the wrong output in the end. This is a very good showcase to illustrate why you prefer a **sigmoid neuron** instead of a **perceptron** in modern neural networks. [Here](https://stats.stackexchange.com/questions/167478/what-is-the-difference-between-sigmoid-neurons-and-stochastic-binary-neurons) is a relevant question from cross-validated concerning the difference of those two types of neurons.
  
  > **Comments:**
  > **username_0**: I did the changes as suggested and indeed the learning process tends to 1. However i have a few questions.    1) why do the weights tend to -1 and -1? if i input 1 1, the output is -1*1 + -1*1 +2 =0. But the answer should be 1 according to AND truth table. It seems that the output is still not correct?  2) could someone explain or provide a link to the importance of learning rate here? i realize that the weights tend more to -1 if i use a small rate like 0.00015. and it gets less accurate when i use a higher learning rate. How do i determine the correct learning rate to use?   Thank you!  
  > **username_3**: It tends towards 0 because 0 (or above) is the threshold for the perceptron to fire. Firing means "output 1" and not firing means "output 0". Using 0 is kind of arbitrary here, but that's what's usually used for perceptrons so I followed this convention. The learning rate just tells you how fast you are learning in each steps. A slower learning rate will require more training cycles but might get you closer to the "perfect" values in the end. There are mechanisms to determine a good constant or even dynamic learning rate, but in the beginning it is mostly trial and error. Hope that helps!  
  
  ---
  
  