Answers.Body: <p>This is the classic question of what structure is or can be. It relates directly to the concepts of generalization, pattern recognition, over-fitting in surface fitting strategies, and learning <em>tabula rasa</em>, Latin for <em>blank slate</em>. The underlying questions are these:</p>
  <blockquote>
  <ol>
  <li><p>How can it be determined whether the organization of data in a set, which appears to correlate well with model <span class="math-container">$\mathbb{A}$</span>, isn't just a random data set that merely appears to have <span class="math-container">$\mathbb{A}$</span> organization.</p>
  </li>
  <li><p>How can it be determined whether the organization of data in a set, which appears to correlate well with model <span class="math-container">$\mathbb{A}$</span>, isn't generated by a phenomenon that generally exhibits organization that correlates well with <span class="math-container">$\mathbb{B}$</span>?</p>
  </li>
  <li><p>How can we determine the interrelationship between <span class="math-container">$\mathbb{A}$</span> and <span class="math-container">$\mathbb{B}$</span>, especially since inclusion and overlap relates to the above question?</p>
  </li>
  </ol>
  </blockquote>
  <p>In the second image in the question, if <span class="math-container">$\mathbb{A}$</span> is defined as clustering of a type the smallest boundaries of which are not breached by instances of another type with at least five members per cluster, then several false positives would occur if the locations of the points were generated with quantum or thermal noise and the model was intended to illuminate clustering. In such a case there may be the appearance of clusters, but proximity would not be indicative of organization.</p>
  <p>In the first image in the question, if <span class="math-container">$\mathbb{A}$</span> is the compound angular projection of the orbital paths of the planets in this solar system, then the image would appear as noise.</p>
  <p>If a third image existed of the moon's sky, the same model would likely produce a false negative if the tolerance of inaccuracy in input was high.</p>
  <p>It is difficult to reliability detect organization in the measured properties of a complex system. The goal of feature extraction, using RBMs or some other approach is to detect patterns. Whether those patterns are organized or merely apparent is a hypothesis that may be strengthened or weakened as the amount of data analyzed increases.</p>
  <p>If an AI system is trained to detect rats but knows nothing of cats and dogs, one cannot assume it will differentiate cats from dogs on the basis of the closer relationship between rodent and cat features than between rodent and dog features unless the AI system was meta-trained or programmed statically with a specific conception of what a feature is.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-12-24T07:13:28.710Z
Answers.LastActivityDate: 2018-12-26T02:41:25.283Z
Answers.LastEditDate: 2020-06-17T09:57:20.710Z
Answers.UserDisplayName: Douglas Daseeco
Body: <p>Consider an image classification problem. Conceptually, we then have some high dimensional space where all the images can be represented as points, and having large enough labeled data set we can build a classifier. But how do we know that our data in this space has some structure? Like this one in two-dimensional case:</p>
  
  <p><a href="https://i.sstatic.net/jxpNO.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/jxpNO.png" alt="enter image description here"></a></p>
  
  <p>If we have a data set with images of, say, cats and dogs, why these two classes are not just uniformly mixed with each other but have some distribution or shape in appropriate space? Why it cannot be like this:</p>
  
  <p><a href="https://i.sstatic.net/kHA5W.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/kHA5W.png" alt="enter image description here"></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-12-19T14:48:24.983Z
Comments.Text: How do you know that data point are shaped together ?
Comments.UserDisplayName: Jérémy Blain
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-12-19T14:57:21.690Z
Comments.Text: I don't know at all. I wonder why is it so in some cases.
Comments.UserDisplayName: Denisof
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-12-20T19:04:36.310Z
Comments.Text: What you are trying to do is create a mapping from the raw image space, where the images are just mixed up randomly, to another space, where the dimensions reflect attributes of the image that allow the images to be clustered in separate groups.  Each layer of the network represents a mapping from one space to another; you hope to get a final layer with a useful dimensions, so the whole network together provides the mapping you want.
Comments.UserDisplayName: antlersoft
Tags: <classification><image-recognition><pattern-recognition><clustering>
Title: How do we know the classification boundaries of the data?
UserDisplayName: Denisof
ThreadText: # How do we know the classification boundaries of the data?
  
  (Asked by: username_0 on 2018-12-19)
  
  Consider an image classification problem. Conceptually, we then have some high dimensional space where all the images can be represented as points, and having large enough labeled data set we can build a classifier. But how do we know that our data in this space has some structure? Like this one in two-dimensional case:
  
  [![enter image description here](https://i.sstatic.net/jxpNO.png)](https://i.sstatic.net/jxpNO.png)
  
  If we have a data set with images of, say, cats and dogs, why these two classes are not just uniformly mixed with each other but have some distribution or shape in appropriate space? Why it cannot be like this:
  
  [![enter image description here](https://i.sstatic.net/kHA5W.png)](https://i.sstatic.net/kHA5W.png)
  
  **Tags:** classification, image-recognition, pattern-recognition, clustering
  
  > **Comments:**
  > **username_1**: How do you know that data point are shaped together ?  
  > **username_0**: I don't know at all. I wonder why is it so in some cases.  
  > **username_2**: What you are trying to do is create a mapping from the raw image space, where the images are just mixed up randomly, to another space, where the dimensions reflect attributes of the image that allow the images to be clustered in separate groups.  Each layer of the network represents a mapping from one space to another; you hope to get a final layer with a useful dimensions, so the whole network together provides the mapping you want.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2018-12-24. Score: 1 | Accepted answer
  
  This is the classic question of what structure is or can be. It relates directly to the concepts of generalization, pattern recognition, over-fitting in surface fitting strategies, and learning *tabula rasa*, Latin for *blank slate*. The underlying questions are these:
  
  > 1. How can it be determined whether the organization of data in a set, which appears to correlate well with model $\mathbb{A}$, isn't just a random data set that merely appears to have $\mathbb{A}$ organization.
  > 2. How can it be determined whether the organization of data in a set, which appears to correlate well with model $\mathbb{A}$, isn't generated by a phenomenon that generally exhibits organization that correlates well with $\mathbb{B}$?
  > 3. How can we determine the interrelationship between $\mathbb{A}$ and $\mathbb{B}$, especially since inclusion and overlap relates to the above question?
  
  In the second image in the question, if $\mathbb{A}$ is defined as clustering of a type the smallest boundaries of which are not breached by instances of another type with at least five members per cluster, then several false positives would occur if the locations of the points were generated with quantum or thermal noise and the model was intended to illuminate clustering. In such a case there may be the appearance of clusters, but proximity would not be indicative of organization.
  
  In the first image in the question, if $\mathbb{A}$ is the compound angular projection of the orbital paths of the planets in this solar system, then the image would appear as noise.
  
  If a third image existed of the moon's sky, the same model would likely produce a false negative if the tolerance of inaccuracy in input was high.
  
  It is difficult to reliability detect organization in the measured properties of a complex system. The goal of feature extraction, using RBMs or some other approach is to detect patterns. Whether those patterns are organized or merely apparent is a hypothesis that may be strengthened or weakened as the amount of data analyzed increases.
  
  If an AI system is trained to detect rats but knows nothing of cats and dogs, one cannot assume it will differentiate cats from dogs on the basis of the closer relationship between rodent and cat features than between rodent and dog features unless the AI system was meta-trained or programmed statically with a specific conception of what a feature is.
  
  
  ---
  
  