Answers.Body: <p>The spatial relationships that you describe would correspond to features, and it's not clear that you need to use a neural network for detecting or discovering these features since you have just described them. Could you instead define a feature extractor that detects the correct patterns and returns you a vector of counts of feature occurrences across the image?</p>
  <p>In fact <a href="https://en.wikipedia.org/wiki/Co-occurrence_matrix" rel="nofollow noreferrer">gray level co-occurrance matrices</a> might be adaptable to this problem. Since there are only four colours, each matrix would be very small, and you would define one matrix for each local spatial configuration. For example, one matrix might encode the number of times that colour <em>x</em> is below colour <em>y</em>. From these you could then extract counts of interest to use for further downstream processing.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-23T06:43:38.053Z
Answers.Text: I think "discovering" them would be proper term here, since I do not know them a priori. Discovering them and then, based on that, doing the classification or clusterisation.
Answers.UserDisplayName: GKozinski
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-22T19:39:42.337Z
Answers.LastActivityDate: 2021-04-22T19:39:42.337Z
Answers.UserDisplayName: Mike NZ
Body: <p>Let's assume I have dataset of image-like 2D samples where values can be divided into few discrete levels (for example 1, 2, 3 and 4) like in the image below, where each color maps different value, from 1 to 4. Number of how many times given color occurs on the picture varies from sample to sample though.</p>
  <p><a href="https://i.sstatic.net/0CS8k.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/0CS8k.jpg" alt="enter image description here" /></a></p>
  <p>I would like to classify these images into different classes but based on the spatial relations of these values between each other (not the values themselves). By spatial relations I mean basically (left, right, up, down), for example:</p>
  <ul>
  <li>If blue is above and to the right of the red</li>
  <li>Another blue is above and to the left of the same red</li>
  <li>Yellow is to the right of one blue (same height)</li>
  <li>One green is below red</li>
  <li>...</li>
  </ul>
  <p>My question is, what algorithm (probably some deep neural network) I should use for that task?
  I would appreciate even just some keywords or clues of what might help.</p>
  
Tags: <deep-learning><classification><image-recognition><deep-neural-networks><knowledge-representation>
Title: What algorithm to use to classify data by spatial relations?
UserDisplayName: GKozinski
ThreadText: # What algorithm to use to classify data by spatial relations?
  
  (Asked by: username_0 on 2021-04-22)
  
  Let's assume I have dataset of image-like 2D samples where values can be divided into few discrete levels (for example 1, 2, 3 and 4) like in the image below, where each color maps different value, from 1 to 4. Number of how many times given color occurs on the picture varies from sample to sample though.
  
  [![enter image description here](https://i.sstatic.net/0CS8k.jpg)](https://i.sstatic.net/0CS8k.jpg)
  
  I would like to classify these images into different classes but based on the spatial relations of these values between each other (not the values themselves). By spatial relations I mean basically (left, right, up, down), for example:
  
  * If blue is above and to the right of the red
  * Another blue is above and to the left of the same red
  * Yellow is to the right of one blue (same height)
  * One green is below red
  * ...
  
  My question is, what algorithm (probably some deep neural network) I should use for that task?
  I would appreciate even just some keywords or clues of what might help.
  
  **Tags:** deep-learning, classification, image-recognition, deep-neural-networks, knowledge-representation
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-04-22. Score: 0
  
  The spatial relationships that you describe would correspond to features, and it's not clear that you need to use a neural network for detecting or discovering these features since you have just described them. Could you instead define a feature extractor that detects the correct patterns and returns you a vector of counts of feature occurrences across the image?
  
  In fact [gray level co-occurrance matrices](https://en.wikipedia.org/wiki/Co-occurrence_matrix) might be adaptable to this problem. Since there are only four colours, each matrix would be very small, and you would define one matrix for each local spatial configuration. For example, one matrix might encode the number of times that colour *x* is below colour *y*. From these you could then extract counts of interest to use for further downstream processing.
  
  > **Comments:**
  > **username_0**: I think "discovering" them would be proper term here, since I do not know them a priori. Discovering them and then, based on that, doing the classification or clusterisation.  
  
  ---
  
  