Answers.Body: <blockquote>
  <p>How to fix the network above to auto-classify XOR data, in unsupervised manner?</p>
  </blockquote>
  <p>This cannot be done, except accidentally.</p>
  <p>Unsupervised learning cannot replace or emulate supervised learning.</p>
  <p>As a thought experiment, consider why you would expect the network to discover XOR, when simply considering outputs rounded to binary, you could equally find AND, OR, NAND, NOR or any of the 16 possible mapping functions from input to output. All of the possible maps are equally valid functions, and there is no reason why a discovered function mapping should become any one of them by preference.</p>
  <p>Unsupervised learning approaches typically find patterns that optimise some measure across the dataset without using labelled data. Clustering is a classic example, and auto-encoding is sometimes considered unsupervised because there is no separate label (although the term <em>self-supervised</em> is also used, because there is still technically a label used in training, it happens to equal the input).</p>
  <p>You cannot use auto-encoding approaches here anyway, because XOR needs to map <span class="math-container">$\{0,1\} \times \{0,1\} \rightarrow \{0,1\}$</span></p>
  <p>You could potentially use a loss function based on how close to a 0 <em>or</em> 1 any output is. That should cause the network to converge to one of the 16 possible binary functions, based on random initialisation. For example, you could use <span class="math-container">$y(1-y)$</span> as the loss.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-25T21:03:09.780Z
Answers.Text: >Unsupervised learning cannot replace or emulate supervised learning.
  Actually, I disagree with it a bit. If you find a pattern/clusters with USL, and then label the pattern/clusters with some label, then you emulate SL a bit.
Answers.UserDisplayName: Daniel Wiczew
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-25T21:09:24.983Z
Answers.Text: @DanielWiczew I think that is strectching definitions, but I suppose it is a matter of taste. I don't think I will include in the answer though, because it won't solve OP's problem.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-26T03:14:29.033Z
Answers.Text: @NeilSlater tks, i now use different fixed init weights for different grouping discovery :)
Answers.UserDisplayName: Dan D.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-25T16:35:11.637Z
Answers.LastActivityDate: 2020-07-25T16:35:11.637Z
Answers.UserDisplayName: Neil Slater
Body: <p>XOR data, without labels:</p>
  <pre><code>[[0,0],[0,1],[1,0],[1,1]]
  </code></pre>
  <p>I'm using this network for auto-classifying XOR data:</p>
  <pre><code>H1  &lt;-- Dense(units=2, activation=relu)    #any activation here
  Z   &lt;-- Dense(units=2, activation=softmax) #softmax for 2 classes of XOR result
  Out &lt;-- Dense(units=2, activation=sigmoid) #sigmoid to return 2 values in (0,1)
  </code></pre>
  <p>There's a logical problem in the network, that is, Z represents 2 classes,
  however, the 2 classes can't be decoded back to 4 samples of XOR data.</p>
  <p>How to fix the network above to auto-classify XOR data, in unsupervised manner?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-07-22T11:18:44.927Z
Comments.Text: Programming issues are off-topic here. You've been a user on this site for a long time, so I recommend you read our on-topic page: https://ai.stackexchange.com/help/on-topic.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-07-23T04:09:42.840Z
Comments.Text: nana, i meant to ask about theory, ie. how to design an autoencoder to group up unsupervised data
Comments.UserDisplayName: Dan D.
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-07-25T13:57:54.530Z
Comments.Text: Ok. Please, next time make sure to clarify that, especially, when you add code to your post.
Comments.UserDisplayName: nbro
Tags: <neural-networks><machine-learning><deep-learning><unsupervised-learning><autoencoders>
Title: What should the output of a neural network that needs to classify in an unsupervised fashion XOR data be?
UserDisplayName: Dan D.
ThreadText: # What should the output of a neural network that needs to classify in an unsupervised fashion XOR data be?
  
  (Asked by: username_0 on 2020-07-22)
  
  XOR data, without labels:
  [[0,0],[0,1],[1,0],[1,1]]
  
  I'm using this network for auto-classifying XOR data:
  H1  <-- Dense(units=2, activation=relu)    #any activation here
  Z   <-- Dense(units=2, activation=softmax) #softmax for 2 classes of XOR result
  Out <-- Dense(units=2, activation=sigmoid) #sigmoid to return 2 values in (0,1)
  
  There's a logical problem in the network, that is, Z represents 2 classes,
  however, the 2 classes can't be decoded back to 4 samples of XOR data.
  How to fix the network above to auto-classify XOR data, in unsupervised manner?
  
  **Tags:** neural-networks, machine-learning, deep-learning, unsupervised-learning, autoencoders
  > **Comments:**
  > **username_1**: Programming issues are off-topic here. You've been a user on this site for a long time, so I recommend you read our on-topic page: https://ai.stackexchange.com/help/on-topic.  
  > **username_0**: nana, i meant to ask about theory, ie. how to design an autoencoder to group up unsupervised data  
  > **username_1**: Ok. Please, next time make sure to clarify that, especially, when you add code to your post.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-07-25. Score: 1 | Accepted answer
  
  > How to fix the network above to auto-classify XOR data, in unsupervised manner?
  
  This cannot be done, except accidentally.
  
  Unsupervised learning cannot replace or emulate supervised learning.
  
  As a thought experiment, consider why you would expect the network to discover XOR, when simply considering outputs rounded to binary, you could equally find AND, OR, NAND, NOR or any of the 16 possible mapping functions from input to output. All of the possible maps are equally valid functions, and there is no reason why a discovered function mapping should become any one of them by preference.
  
  Unsupervised learning approaches typically find patterns that optimise some measure across the dataset without using labelled data. Clustering is a classic example, and auto-encoding is sometimes considered unsupervised because there is no separate label (although the term *self-supervised* is also used, because there is still technically a label used in training, it happens to equal the input).
  
  You cannot use auto-encoding approaches here anyway, because XOR needs to map $\{0,1\} \times \{0,1\} \rightarrow \{0,1\}$
  
  You could potentially use a loss function based on how close to a 0 *or* 1 any output is. That should cause the network to converge to one of the 16 possible binary functions, based on random initialisation. For example, you could use $y(1-y)$ as the loss.
  
  > **Comments:**
  > **username_3**: >Unsupervised learning cannot replace or emulate supervised learning. Actually, I disagree with it a bit. If you find a pattern/clusters with USL, and then label the pattern/clusters with some label, then you emulate SL a bit.  
  > **username_2**: @DanielWiczew I think that is strectching definitions, but I suppose it is a matter of taste. I don't think I will include in the answer though, because it won't solve OP's problem.  
  > **username_0**: @NeilSlater tks, i now use different fixed init weights for different grouping discovery :)  
  
  ---
  
  