Answers.Body: <p>Kohonen networks by definition are single layer FCNN's, but what differentiates them from others is their unsupervised training procedure.   </p>
  
  <p>This procedure is a function of the input, the weights and some hyperparameters. This means if you have a multilayer network you could train only the final layers weights using this procedure. Think of it this way, let <span class="math-container">$f$</span> be the final layer, and <span class="math-container">$g$</span> be the composition of all other layers, such that the whole network <span class="math-container">$N(x) = f \circ g(x)$</span>. Using kohonens training procedure, you could learn <span class="math-container">$f$</span>'s weights where the input is <span class="math-container">$g(x)$</span> rather than x. But then how would you learn <span class="math-container">$g$</span>'s weights?  </p>
  
  <p>So you could do this by iterative learning. let <span class="math-container">$N$</span> be a 2 layer network: <span class="math-container">$N = f_2 \circ f_1$</span>. first learn <span class="math-container">$f_1$</span> using the single layer procedure, and now you could learn <span class="math-container">$f_2$</span> in the same manner, except the input features its trying to cluster would be <span class="math-container">$f_1(x)$</span> rather than <span class="math-container">$x$</span>.  </p>
  
  <p>There does exist cons to this procedure though: Using a kohonen network at each layer will try to cluster it to the best of its abilities, not so that its final composition will be best (a common problem in many optimization procedures that arent end-to-end), but that its current representation is. This may lead to non-optimal results, not achieving that deep representation that you're looking for that you could achieve in autoencoders or other deeper unsupervised models.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-17T19:22:58.877Z
Answers.Text: I also guessed you can not use kohonnen with the normal layers, because the mertic of those output is not like the from original data, means it is not differomorf.  I v made small test and watched that for example in autoencoders middle layer.  I v asked, is it generally known multiple knns -> means not.
Answers.UserDisplayName: user8426627
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-17T22:10:58.453Z
Answers.Text: @DuttaA thats kinda my point: since you can learn f1 then f2, f2 will add almost no benefit because its optimized for the singular goal at each layer rather than for a deeper end-to-end representation you get in autoencoder type models
Answers.UserDisplayName: mshlis
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-17T18:16:30.500Z
Answers.LastActivityDate: 2019-07-17T18:16:30.500Z
Answers.UserDisplayName: mshlis
Body: <p>The Kohonen network is one fully connected layer, which clusters the input into classes by a given metric. However, the one layer does not allow to operate with complex relations, that's why deep learning is usually used.</p>
  
  <p>Is it possible then to make multi-layered Kohonen network? </p>
  
  <p>AFAIK, the output of the first layer is already cluster flags, so the activation function on the non-last layers must be different from the original Kohonnen definition?</p>
  
Tags: <neural-networks><unsupervised-learning>
Title: Is a multi-layer Kohonen network possible?
UserDisplayName: user8426627
ThreadText: # Is a multi-layer Kohonen network possible?
  
  (Asked by: username_0 on 2019-07-17)
  
  The Kohonen network is one fully connected layer, which clusters the input into classes by a given metric. However, the one layer does not allow to operate with complex relations, that's why deep learning is usually used.
  
  Is it possible then to make multi-layered Kohonen network?
  
  AFAIK, the output of the first layer is already cluster flags, so the activation function on the non-last layers must be different from the original Kohonnen definition?
  
  **Tags:** neural-networks, unsupervised-learning
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-07-17. Score: 1
  
  Kohonen networks by definition are single layer FCNN's, but what differentiates them from others is their unsupervised training procedure.
  
  This procedure is a function of the input, the weights and some hyperparameters. This means if you have a multilayer network you could train only the final layers weights using this procedure. Think of it this way, let $f$ be the final layer, and $g$ be the composition of all other layers, such that the whole network $N(x) = f \circ g(x)$. Using kohonens training procedure, you could learn $f$'s weights where the input is $g(x)$ rather than x. But then how would you learn $g$'s weights?
  
  So you could do this by iterative learning. let $N$ be a 2 layer network: $N = f\_2 \circ f\_1$. first learn $f\_1$ using the single layer procedure, and now you could learn $f\_2$ in the same manner, except the input features its trying to cluster would be $f\_1(x)$ rather than $x$.
  
  There does exist cons to this procedure though: Using a kohonen network at each layer will try to cluster it to the best of its abilities, not so that its final composition will be best (a common problem in many optimization procedures that arent end-to-end), but that its current representation is. This may lead to non-optimal results, not achieving that deep representation that you're looking for that you could achieve in autoencoders or other deeper unsupervised models.
  
  > **Comments:**
  > **username_0**: I also guessed you can not use kohonnen with the normal layers, because the mertic of those output is not like the from original data, means it is not differomorf.  I v made small test and watched that for example in autoencoders middle layer.  I v asked, is it generally known multiple knns -> means not.  
  > **username_1**: @DuttaA thats kinda my point: since you can learn f1 then f2, f2 will add almost no benefit because its optimized for the singular goal at each layer rather than for a deeper end-to-end representation you get in autoencoder type models  
  
  ---
  
  