Answers.Body: <p>The perceptron has a step activation. This does not.</p>
  <p>But at a deeper level, what you are showing is that, if we define the neuron's activation function to be XOR (essentially what the mod 2 addition means), then it can compute XOR. That is a vacuous statement. It's like saying a neuron can compute any function <em>f</em>, as long as we define it's activation as <em>f</em>. Obvious, but useless.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-02T15:07:15.013Z
Answers.Text: "That is a vacuous statement.  It's like saying a neuron can compute any function f, as long as we define it's activation as f"
  
  But notice that when we ask if "a neuron can compute any function f" we are speaking of a multivariate function (all inputs), while the activation function takes a single input . Not so vacuous. That is, given an arbitrary $g(x_1, x_2 ... x_n)$, is it true that we can represent it by some $g = f ( h(x_1, x_2 , x_n))$ where h is a linear combination and f is an arbitrary scalar function ?
Answers.UserDisplayName: leonbloy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-31T16:47:16.410Z
Answers.LastActivityDate: 2024-01-01T02:45:39.837Z
Answers.LastEditDate: 2024-01-01T02:45:39.837Z
Answers.UserDisplayName: chessprogrammer
Answers.Body: <p>As far as I remember, the activation functions in our university lecture (which we called transfer function as well) were always (non-strictly) ~~monotonous~~ monotonic (thanks to A.R. for the correction). Modulo is not a monotonic function.</p>
  <p>The single layer perceptron (which we also called connectionist neuron) is indeed incapable of XOR-classification if we have a monotonic activation function because in that case the binary classification is one straight line or hyperplane separator in space (ternary would be two parallel hyperplanes, etc.), and the activation function is necessarily a heaviside function, an edge detector.</p>
  <p>We did not formally define XOR classification but we considered it to be the capability of dividing space such that 4 disjoint subsets of space are assigned two classes, diagonal subsets belong to the same class. (It would make sense to define XOR as 4 quadrants in a 2D space.) Between every two subsets of the same class, there is one subset of the opposite class. When two same-class subsets are in one half then a third subset also will be in the half. We need  at least a second linear classifier in addition to separate both classes in each half.</p>
  <p>By the way, modulo is not a binary classifier and therefore cannot be considered to be an XOR or other binary classification. Binary classifiers produce binary output for real-valued input vectors and &quot;mod x&quot; certainly has a ~~domain~~ real-valued range (thanks to John Madden for the correction) from 0 (inclusive) to &quot;x&quot; (exclusive).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-03T19:55:00.533Z
Answers.Text: I think the word you're thinking of is _monotonic._ "Monotonous" means "boring."
Answers.UserDisplayName: OpenAI was the last straw
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-03T21:11:58.647Z
Answers.Text: Re your final para: but since the OP is referring to mod 2, then it has a range (not domain) of 0 and 1,  and is hence indeed binary. The issue is indeed the nonmonotonicity of the activation function as you described earlier.
Answers.UserDisplayName: John Madden
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-04T22:02:32.487Z
Answers.Text: Yes, right. Funny thing, "monotonous" and "monotonic" are the same word in German.
Answers.UserDisplayName: ChrisoLosoph
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-03T19:00:35.497Z
Answers.LastActivityDate: 2024-01-04T22:09:05.333Z
Answers.LastEditDate: 2024-01-04T22:09:05.333Z
Answers.UserDisplayName: ChrisoLosoph
Body: <pre><code>def activation_function(number):
    if number%2:
      return 1
    return 0
  
  weights = [1, 1]
  for x in range(2):
    for y in range(2):
      print(f&quot;{x}, {y} = {activation_function(weights[0] * x + weights[1] * y)}&quot;)
  </code></pre>
  <p>Output:</p>
  <pre><code>0, 0 = 0
  0, 1 = 1
  1, 0 = 1
  1, 1 = 0
  </code></pre>
  <p>What am I missing here?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-12-31T16:23:18.703Z
Comments.Text: The perceptron uses the step function https://en.wikipedia.org/wiki/Perceptron
Comments.UserDisplayName: Jaume Oliver Lafont
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-01-02T18:32:42.677Z
Comments.Text: Sidenote: `if number%2: return 1; return 0` is equivalent to `return (number % 2)`.
Comments.UserDisplayName: Stef
Tags: <perceptron><xor-problem><single-layer-perceptron>
Title: Why is it believed that a single-layer perceptron can't solve XOR? Doesn't this example disprove that?
UserDisplayName: Jaroslav Tavgen
ThreadText: # Why is it believed that a single-layer perceptron can't solve XOR? Doesn't this example disprove that?
  
  (Asked by: username_0 on 2023-12-31)
  
  ```
  def activation_function(number):
    if number%2:
      return 1
    return 0
  
  weights = [1, 1]
  for x in range(2):
    for y in range(2):
      print(f"{x}, {y} = {activation_function(weights[0] * x + weights[1] * y)}")
  
  ```
  
  Output:
  
  ```
  0, 0 = 0
  0, 1 = 1
  1, 0 = 1
  1, 1 = 0
  
  ```
  
  What am I missing here?
  
  **Tags:** perceptron, xor-problem, single-layer-perceptron
  
  > **Comments:**
  > **username_1**: The perceptron uses the step function https://en.wikipedia.org/wiki/Perceptron  
  > **username_2**: Sidenote: `if number%2: return 1; return 0` is equivalent to `return (number % 2)`.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2023-12-31. Score: 28
  
  The perceptron has a step activation. This does not.
  
  But at a deeper level, what you are showing is that, if we define the neuron's activation function to be XOR (essentially what the mod 2 addition means), then it can compute XOR. That is a vacuous statement. It's like saying a neuron can compute any function *f*, as long as we define it's activation as *f*. Obvious, but useless.
  
  > **Comments:**
  > **username_4**: "That is a vacuous statement.  It's like saying a neuron can compute any function f, as long as we define it's activation as f"  But notice that when we ask if "a neuron can compute any function f" we are speaking of a multivariate function (all inputs), while the activation function takes a single input . Not so vacuous. That is, given an arbitrary $g(x_1, x_2 ... x_n)$, is it true that we can represent it by some $g = f ( h(x_1, x_2 , x_n))$ where h is a linear combination and f is an arbitrary scalar function ?  
  
  ---
  
  ### Answer by username_5 on 2024-01-03. Score: 0
  
  As far as I remember, the activation functions in our university lecture (which we called transfer function as well) were always (non-strictly) ~~monotonous~~ monotonic (thanks to A.R. for the correction). Modulo is not a monotonic function.
  
  The single layer perceptron (which we also called connectionist neuron) is indeed incapable of XOR-classification if we have a monotonic activation function because in that case the binary classification is one straight line or hyperplane separator in space (ternary would be two parallel hyperplanes, etc.), and the activation function is necessarily a heaviside function, an edge detector.
  
  We did not formally define XOR classification but we considered it to be the capability of dividing space such that 4 disjoint subsets of space are assigned two classes, diagonal subsets belong to the same class. (It would make sense to define XOR as 4 quadrants in a 2D space.) Between every two subsets of the same class, there is one subset of the opposite class. When two same-class subsets are in one half then a third subset also will be in the half. We need at least a second linear classifier in addition to separate both classes in each half.
  
  By the way, modulo is not a binary classifier and therefore cannot be considered to be an XOR or other binary classification. Binary classifiers produce binary output for real-valued input vectors and "mod x" certainly has a ~~domain~~ real-valued range (thanks to username_7 username_7 for the correction) from 0 (inclusive) to "x" (exclusive).
  
  > **Comments:**
  > **username_6**: I think the word you're thinking of is _monotonic._ "Monotonous" means "boring."  
  > **username_7**: Re your final para: but since the OP is referring to mod 2, then it has a range (not domain) of 0 and 1,  and is hence indeed binary. The issue is indeed the nonmonotonicity of the activation function as you described earlier.  
  > **username_5**: Yes, right. Funny thing, "monotonous" and "monotonic" are the same word in German.  
  
  ---
  
  