Answers.Body: <p>SOM (Self-Organinizing Map) is a type of artificial neural network (ANN), introduced by the Finnish professor Teuvo Kohonen in the 1980s, that is trained using unsupervised learning to produce a low-dimensional, discretized representation of the input space of the training samples, called a map, and is therefore a method to do dimensionality reduction.</p>
  <p>SOM produces a mapping from a multidimensional input space onto a lattice of clusters, i.e. neurons, in a way that preserves their topology, so that
  neighboring neurons respond to “similar” input patterns.</p>
  <p>It uses three basic processes:</p>
  <ul>
  <li>Competition</li>
  <li>Cooperation</li>
  <li>Adaptation</li>
  </ul>
  <p><em>In competition,</em> each neuron is assigned a weight vector with the same
  dimensionality d as the input space. Any given input pattern is compared to the weight vector of each neuron and the closest neuron is declared the winner.</p>
  <p><em>In cooperation</em>, the activation of the winning neuron is spread to neurons in its immediate neighborhood, and as a result this allows topologically close neurons to become sensitive to similar patterns. The size of the neighborhood is initially large, but shrinks over time, where an initially large neighborhood promotes a topology-preserving mapping and smaller neighborhoods allows neurons to specialize in the latter stages of training.</p>
  <p><em>In adaptation</em>, the winner neuron and its topological neighbors are adapted to make their weight vectors more similar to the input pattern that caused the activation.</p>
  <p>So, a Self-Organizing Map (SOM) is to encode a large set of input vectors <span class="math-container">$\textbf{x}$</span> by finding a smaller set of “representatives” or
  “prototypes” or “code-book vectors” <span class="math-container">$\textbf{w}$</span> that provide a good approximation to the original input space. This is the basic idea of vector quantization theory, the motivation of which is dimensionality reduction or data compression. Performing a gradient descent style minimization on SOM's loss function (eg. the sum of the Euclidean distances between the input sample and each neuron) does lead to the SOM weight update algorithm, which confirms that it is generating the best possible discrete low dimensional approximation to the input space (at least assuming it does not get trapped in a local minimum of the error function).</p>
  <p><strong>To answer your question, you should take into consideration that the Dimensionality reduction takes place in fields that deal with large numbers of observations and/or large numbers of variables. Thus, SOM helps finding good &quot;prototypes&quot;, in a way that each input pattern belongs to exactly one of them. As a result, the training instances are mapped to the training &quot;prototypes&quot; and the whole training set is mapped to a new one with less instances.</strong></p>
  <p>In addition, the &quot;prototypes&quot; neurons resulted by SOM can often be used as good centers in RBF networks or to classify patterns with the LVQ family algorithms.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-04T15:24:48.007Z
Answers.Text: Good explanation about SOM but I could not find where you answered my ques
Answers.UserDisplayName: Sivaram Rasathurai
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-04T15:25:02.380Z
Answers.Text: can you bold those lines
Answers.UserDisplayName: Sivaram Rasathurai
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-04T15:20:45.910Z
Answers.LastActivityDate: 2020-10-04T15:33:03.547Z
Answers.LastEditDate: 2020-10-04T15:33:03.547Z
Answers.UserDisplayName: ddaedalus
Body: <p>We have n dimension input for SOM and the output 2-D clusters. How does it happen?</p>
  
Tags: <machine-learning><unsupervised-learning><self-organizing-map>
Title: How does dimensionality reduction occur in Self organizing Map (SOM)?
UserDisplayName: Sivaram Rasathurai
ThreadText: # How does dimensionality reduction occur in Self organizing Map (SOM)?
  
  (Asked by: username_0 on 2020-10-04)
  
  We have n dimension input for SOM and the output 2-D clusters. How does it happen?
  
  **Tags:** machine-learning, unsupervised-learning, self-organizing-map
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-10-04. Score: 3 | Accepted answer
  
  SOM (Self-Organinizing Map) is a type of artificial neural network (ANN), introduced by the Finnish professor Teuvo Kohonen in the 1980s, that is trained using unsupervised learning to produce a low-dimensional, discretized representation of the input space of the training samples, called a map, and is therefore a method to do dimensionality reduction.
  
  SOM produces a mapping from a multidimensional input space onto a lattice of clusters, i.e. neurons, in a way that preserves their topology, so that
  neighboring neurons respond to “similar” input patterns.
  
  It uses three basic processes:
  
  * Competition
  * Cooperation
  * Adaptation
  
  *In competition,* each neuron is assigned a weight vector with the same
  dimensionality d as the input space. Any given input pattern is compared to the weight vector of each neuron and the closest neuron is declared the winner.
  
  *In cooperation*, the activation of the winning neuron is spread to neurons in its immediate neighborhood, and as a result this allows topologically close neurons to become sensitive to similar patterns. The size of the neighborhood is initially large, but shrinks over time, where an initially large neighborhood promotes a topology-preserving mapping and smaller neighborhoods allows neurons to specialize in the latter stages of training.
  
  *In adaptation*, the winner neuron and its topological neighbors are adapted to make their weight vectors more similar to the input pattern that caused the activation.
  
  So, a Self-Organizing Map (SOM) is to encode a large set of input vectors $\textbf{x}$ by finding a smaller set of “representatives” or
  “prototypes” or “code-book vectors” $\textbf{w}$ that provide a good approximation to the original input space. This is the basic idea of vector quantization theory, the motivation of which is dimensionality reduction or data compression. Performing a gradient descent style minimization on SOM's loss function (eg. the sum of the Euclidean distances between the input sample and each neuron) does lead to the SOM weight update algorithm, which confirms that it is generating the best possible discrete low dimensional approximation to the input space (at least assuming it does not get trapped in a local minimum of the error function).
  
  **To answer your question, you should take into consideration that the Dimensionality reduction takes place in fields that deal with large numbers of observations and/or large numbers of variables. Thus, SOM helps finding good "prototypes", in a way that each input pattern belongs to exactly one of them. As a result, the training instances are mapped to the training "prototypes" and the whole training set is mapped to a new one with less instances.**
  
  In addition, the "prototypes" neurons resulted by SOM can often be used as good centers in RBF networks or to classify patterns with the LVQ family algorithms.
  
  > **Comments:**
  > **username_0**: Good explanation about SOM but I could not find where you answered my ques  
  > **username_0**: can you bold those lines  
  
  ---
  
  