Answers.Body: <h2>Neural Network equivalents that is not (vanilla) feed forward Neural Nets:</h2>
  
  <p>Neural net structures such as Recurrent Neural Nets (RNNs) and Convolutional Neural Nets (CNNs), and different architectures within those are good examples.</p>
  
  <p>Examples of different architectures within RNNs would would be: Long Short Term Memory (LSTM) or Gated Recurrent Unit (GRU). Both of these are well described in Colah's blog post on <a href="http://colah.github.io/posts/2015-08-Understanding-LSTMs/" rel="nofollow noreferrer">Understanding LSTMs</a></p>
  
  <h2>What are some alternative information processing system beside neural network</h2>
  
  <p>There are sooo many structures. From the top of my head: (Restricted) Boltzmann machine, auto encoders, monte carlo method and radial basis networks to name a <em>few</em>.</p>
  
  <p>You can check out Goodfellow's <a href="http://www.deeplearningbook.org/" rel="nofollow noreferrer">Deep learning</a>-book that is free online and get the gist of all the structures I mentioned here (most parts requires a bit of math knowledge, but he also writes about them quite intuitively).</p>
  
  <p>For Recurrent Neural Nets I recommend Colah's blog post on <a href="http://colah.github.io/posts/2015-08-Understanding-LSTMs/" rel="nofollow noreferrer">Understanding LSTMs</a></p>
  
  <h2>Is there any system in which the topology of a neural network is variable?</h2>
  
  <p>Depends on what you mean with the <em>topology</em> of a neural network:</p>
  
  <p>I think in the common meaning of topology when talking about Neural Networks is the way in which neurons are connected to form a network, varying in structure as it runs and learns. If this is what you men then the answer, in short, is yes. In multiple ways actually. On the other hand, if you mean in the mathematical sense, this answer would become a book that I wouldn't feel confortable writing. So I'll assume you mean the first.</p>
  
  <p>We often do "regularization", both on vanilla NN and other structures. One of these regularization techniques are called <a href="https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf" rel="nofollow noreferrer">dropout</a>, which would randomly remove connections from the network as it is training (to prevent something called <a href="https://en.m.wikipedia.org/wiki/Overfitting" rel="nofollow noreferrer">overfitting</a>, which I'm not gonna go into in this post).</p>
  
  <p>Another example for another way would be on the Recurrent Neural Network. They deal with time series, and are equipped for dealing with timeseries of different lengths (thus, "varying structure").</p>
  
  <h2>Does it exist neural net systems where complex numbers are used?</h2>
  
  <p>Yes, there are many papers on complex number machine learning structures. A quick google should give you loads of results. For example: DeepMind has a paper on <a href="https://arxiv.org/pdf/1602.03032.pdf" rel="nofollow noreferrer">
  Associative Long Short-Term Memory</a> which explores the use of complex values for an "associative memory".</p>
  
  <h2>Links:</h2>
  
  <p>Goodfellow's Deep Learning-book: <a href="http://www.deeplearningbook.org/" rel="nofollow noreferrer">deeplearningbook.org</a></p>
  
  <p>Colah's blogpost on RNN's: <a href="http://colah.github.io/posts/2015-08-Understanding-LSTMs/" rel="nofollow noreferrer">colah.github.io</a></p>
  
  <p>Paper on DeepMinds Associative LSTM: <a href="https://arxiv.org/pdf/1602.03032.pdf" rel="nofollow noreferrer">arxiv:1602.03032</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-03-28T09:33:45.573Z
Answers.LastActivityDate: 2018-05-25T19:28:56.193Z
Answers.LastEditDate: 2018-05-25T19:28:56.193Z
Answers.UserDisplayName: Andreas Storvik Strauman
Answers.Body: <p>To answer the title, there are many other machine learning models, but neural networks work particularly well for some difficult problems (image classification, speech recognition) which is one of the reasons they have gained popularity.</p>
  
  <p>Two particularly simple models are the decision tree and the perceptron. These are rather simple models, but they both have redeemable qualities. A decision tree is useful as it provides a model that is easily understood, while a perceptron is fairly quick and works well for linearly separable data. Another, more advanced, model is the Support Vector Machine.</p>
  
  <blockquote>
    <p>For example, is there any system in which the topology of a neural network is variable? </p>
  </blockquote>
  
  <p>Yes, there are many such systems where the topology of the neural network is dynamic throughout training. An entire class of methods labeled TWEANNs are designed to evolve the topology of the networks, one such algorithm is NeuroEvolution of Augmenting Topologies, NEAT (and it's descendants rtNEAT, hyperNEAT, ...).</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-03-27T22:14:51.143Z
Answers.LastActivityDate: 2018-03-27T22:14:51.143Z
Answers.UserDisplayName: Andrew Butler
Answers.Body: <p>A very popular choice are <a href="https://en.wikipedia.org/wiki/Hidden_Markov_model" rel="nofollow noreferrer">Hidden Markov Models</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-28T13:41:42.283Z
Answers.LastActivityDate: 2018-05-28T13:41:42.283Z
Answers.UserDisplayName: pcko1
Body: <p><a href="https://i.sstatic.net/ejFBN.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ejFBN.png" alt="enter image description here"></a>  </p>
  
  <p>Feedforward or multilayered neural networks, like the one in the image above, are usually characterized by the fact that all weighted connections can be represented as a continuous real number. Furthermore, each node in a layer is connected to every other node in the previous and successive layers.</p>
  
  <p><em>Are there any other information processing models other than FFNNs or MLPs?</em> For example, is there any system in which the topology of a neural network is variable? Or a system in which the connections between nodes are not real numbers?</p>
  
Tags: <neural-networks><deep-learning><reference-request><model-request>
Title: What are some information processing models besides MLPs?
UserDisplayName: user289661
ThreadText: # What are some information processing models besides MLPs?
  
  (Asked by: username_0 on 2017-03-07)
  
  [![enter image description here](https://i.sstatic.net/ejFBN.png)](https://i.sstatic.net/ejFBN.png)
  
  Feedforward or multilayered neural networks, like the one in the image above, are usually characterized by the fact that all weighted connections can be represented as a continuous real number. Furthermore, each node in a layer is connected to every other node in the previous and successive layers.
  
  *Are there any other information processing models other than FFNNs or MLPs?* For example, is there any system in which the topology of a neural network is variable? Or a system in which the connections between nodes are not real numbers?
  
  **Tags:** neural-networks, deep-learning, reference-request, model-request
  
  ---
  
  ## 3 answers
  
  ### Answer by username_1 on 2018-03-28. Score: 3
  
  ## Neural Network equivalents that is not (vanilla) feed forward Neural Nets:
  
  Neural net structures such as Recurrent Neural Nets (RNNs) and Convolutional Neural Nets (CNNs), and different architectures within those are good examples.
  
  Examples of different architectures within RNNs would would be: Long Short Term Memory (LSTM) or Gated Recurrent Unit (GRU). Both of these are well described in Colah's blog post on [Understanding LSTMs](http://colah.github.io/posts/2015-08-Understanding-LSTMs/)
  
  ## What are some alternative information processing system beside neural network
  
  There are sooo many structures. From the top of my head: (Restricted) Boltzmann machine, auto encoders, monte carlo method and radial basis networks to name a *few*.
  
  You can check out Goodfellow's [Deep learning](http://www.deeplearningbook.org/)-book that is free online and get the gist of all the structures I mentioned here (most parts requires a bit of math knowledge, but he also writes about them quite intuitively).
  
  For Recurrent Neural Nets I recommend Colah's blog post on [Understanding LSTMs](http://colah.github.io/posts/2015-08-Understanding-LSTMs/)
  
  ## Is there any system in which the topology of a neural network is variable?
  
  Depends on what you mean with the *topology* of a neural network:
  
  I think in the common meaning of topology when talking about Neural Networks is the way in which neurons are connected to form a network, varying in structure as it runs and learns. If this is what you men then the answer, in short, is yes. In multiple ways actually. On the other hand, if you mean in the mathematical sense, this answer would become a book that I wouldn't feel confortable writing. So I'll assume you mean the first.
  
  We often do "regularization", both on vanilla NN and other structures. One of these regularization techniques are called [dropout](https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf), which would randomly remove connections from the network as it is training (to prevent something called [overfitting](https://en.m.wikipedia.org/wiki/Overfitting), which I'm not gonna go into in this post).
  
  Another example for another way would be on the Recurrent Neural Network. They deal with time series, and are equipped for dealing with timeseries of different lengths (thus, "varying structure").
  
  ## Does it exist neural net systems where complex numbers are used?
  
  Yes, there are many papers on complex number machine learning structures. A quick google should give you loads of results. For example: DeepMind has a paper on [Associative Long Short-Term Memory](https://arxiv.org/pdf/1602.03032.pdf) which explores the use of complex values for an "associative memory".
  
  ## Links:
  
  Goodfellow's Deep Learning-book: [deeplearningbook.org](http://www.deeplearningbook.org/)
  
  Colah's blogpost on RNN's: [colah.github.io](http://colah.github.io/posts/2015-08-Understanding-LSTMs/)
  
  Paper on DeepMinds Associative LSTM: [arxiv:1602.03032](https://arxiv.org/pdf/1602.03032.pdf)
  
  
  ---
  
  ### Answer by username_2 on 2018-03-27. Score: 0
  
  To answer the title, there are many other machine learning models, but neural networks work particularly well for some difficult problems (image classification, speech recognition) which is one of the reasons they have gained popularity.
  
  Two particularly simple models are the decision tree and the perceptron. These are rather simple models, but they both have redeemable qualities. A decision tree is useful as it provides a model that is easily understood, while a perceptron is fairly quick and works well for linearly separable data. Another, more advanced, model is the Support Vector Machine.
  
  > For example, is there any system in which the topology of a neural network is variable?
  
  Yes, there are many such systems where the topology of the neural network is dynamic throughout training. An entire class of methods labeled TWEANNs are designed to evolve the topology of the networks, one such algorithm is NeuroEvolution of Augmenting Topologies, NEAT (and it's descendants rtNEAT, hyperNEAT, ...).
  
  
  ---
  
  ### Answer by username_3 on 2018-05-28. Score: -1
  
  A very popular choice are [Hidden Markov Models](https://en.wikipedia.org/wiki/Hidden_Markov_model).
  
  
  ---
  
  