Answers.Body: <p>The following articles</p>
  
  <ul>
  <li><p><a href="https://arxiv.org/abs/q-bio/0611072" rel="nofollow noreferrer">Ising models for networks of real neurons</a> (2006) by Gasper Tkacik et al.</p></li>
  <li><p><a href="https://journals.aps.org/pre/abstract/10.1103/PhysRevE.97.032119" rel="nofollow noreferrer">Deep neural networks for direct, featureless learning through observation: The case of two-dimensional spin models</a> (2018) by Kyle Mills et al.</p></li>
  <li><p><a href="https://arxiv.org/abs/1706.08466" rel="nofollow noreferrer">Inverse Ising inference by combining Ornstein-Zernike theory with deep learning</a> (2017) by Soma Turi, Alpha A. Lee et al.</p></li>
  <li><p><a href="https://arxiv.org/abs/1906.10935" rel="nofollow noreferrer">Neural-network based general method for statistical mechanics on sparse systems</a> (2019) by Feng Pan, Hai-Jun Zhou, Pan Zhang et al.</p></li>
  </ul>
  
  <p>may help you understand the "relationship" between Ising models and DNN, assuming you know what the Ising model is and what a DNN is, the similarity should be fairly intuitive to you. </p>
  
  <p>The <strong>Ising model</strong> is a sort of floating soup of ferromagnetic particles each generating their own small magnetic field either working against or with their neighbor. When many of the particles aline, they create an aligned field we refer to as a dipole moment in magnetism, while in a <strong>DNN</strong> we refer to the joined effort of a few entities working to cause a larger effect in another entity an 'activation function'. In a fully connected DNN, where the Euclidean distance weights the connections and the nodes are initialized with a certain magnetic polarity in relation to the axis of the magnetic field it generates, the network would be an almost exact representation of the reality of what the Ising model seeks to simplify. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-13T14:30:01.223Z
Answers.Text: Thank you ! In Ising model, when its partition function has singularity, which means there is a phase transition for the spin system, lots of interesting things can be said about the system. For DNN, what is the corresponding properties when its partition function is zero or singular ? Any papers on this ? Thank you again.
Answers.UserDisplayName: david
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-13T17:43:50.760Z
Answers.Text: Very little research goes into examine DNN like this. Phase changes would be similar to decision boundaries of the activation functions surounding a location, similar to how a small disturbance in water causes a change in pressure at a location and cause the first ice crystal to form which can cause the chain reaction of other ice crystals forming. I'll edit in some papers at the end to check if it adds to the answer.
Answers.UserDisplayName: Michael Hearn
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-15T19:29:18.217Z
Answers.Text: I am from physics background. I am interested in two types of questions. One is, how to use existing physics results of Ising model to DNN. Another question I am interested is, how to determine the interaction coefficient of spin model if we know its partition function. Do you know, any DNN research on the 2nd type of question ? Thank you.
Answers.UserDisplayName: david
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-15T20:48:40.567Z
Answers.Text: My intuition is that if you have the partition function and existing Ising data there would be an algorithms for seeding a Deep Neural network in an ideal way so as to allow for learning the interaction coefficients in the network much faster. Knowing the function a DNN is attempting to learn/emulate allows for the resources of a NN to be used in a more optimal, by elimination of unnecessary variables/features. I'll keep my eye out for such research.
Answers.UserDisplayName: Michael Hearn
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-09T05:15:31.807Z
Answers.LastActivityDate: 2019-11-13T17:52:39.857Z
Answers.LastEditDate: 2019-11-13T17:52:39.857Z
Answers.UserDisplayName: Michael Hearn
Body: <p>I am looking for a book or paper which clearly explains the relationship between <a href="https://en.wikipedia.org/wiki/Hopfield_network" rel="nofollow noreferrer">Ising models</a> and deep neural networks.</p>
  
  <p>Can anyone provide any references?</p>
  
Tags: <neural-networks><comparison><reference-request><papers><books>
Title: Which books or papers clearly explain the relation between Ising models and deep neural networks?
UserDisplayName: david
ThreadText: # Which books or papers clearly explain the relation between Ising models and deep neural networks?
  
  (Asked by: username_0 on 2019-11-08)
  
  I am looking for a book or paper which clearly explains the relationship between [Ising models](https://en.wikipedia.org/wiki/Hopfield_network) and deep neural networks.
  
  Can anyone provide any references?
  
  **Tags:** neural-networks, comparison, reference-request, papers, books
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-11-09. Score: 4
  
  The following articles
  
  * [Ising models for networks of real neurons](https://arxiv.org/abs/q-bio/0611072) (2006) by Gasper Tkacik et al.
  * [Deep neural networks for direct, featureless learning through observation: The case of two-dimensional spin models](https://journals.aps.org/pre/abstract/10.1103/PhysRevE.97.032119) (2018) by Kyle Mills et al.
  * [Inverse Ising inference by combining Ornstein-Zernike theory with deep learning](https://arxiv.org/abs/1706.08466) (2017) by Soma Turi, Alpha A. Lee et al.
  * [Neural-network based general method for statistical mechanics on sparse systems](https://arxiv.org/abs/1906.10935) (2019) by Feng Pan, Hai-Jun Zhou, Pan Zhang et al.
  
  may help you understand the "relationship" between Ising models and DNN, assuming you know what the Ising model is and what a DNN is, the similarity should be fairly intuitive to you.
  
  The **Ising model** is a sort of floating soup of ferromagnetic particles each generating their own small magnetic field either working against or with their neighbor. When many of the particles aline, they create an aligned field we refer to as a dipole moment in magnetism, while in a **DNN** we refer to the joined effort of a few entities working to cause a larger effect in another entity an 'activation function'. In a fully connected DNN, where the Euclidean distance weights the connections and the nodes are initialized with a certain magnetic polarity in relation to the axis of the magnetic field it generates, the network would be an almost exact representation of the reality of what the Ising model seeks to simplify.
  
  > **Comments:**
  > **username_0**: Thank you ! In Ising model, when its partition function has singularity, which means there is a phase transition for the spin system, lots of interesting things can be said about the system. For DNN, what is the corresponding properties when its partition function is zero or singular ? Any papers on this ? Thank you again.  
  > **username_1**: Very little research goes into examine DNN like this. Phase changes would be similar to decision boundaries of the activation functions surounding a location, similar to how a small disturbance in water causes a change in pressure at a location and cause the first ice crystal to form which can cause the chain reaction of other ice crystals forming. I'll edit in some papers at the end to check if it adds to the answer.  
  > **username_0**: I am from physics background. I am interested in two types of questions. One is, how to use existing physics results of Ising model to DNN. Another question I am interested is, how to determine the interaction coefficient of spin model if we know its partition function. Do you know, any DNN research on the 2nd type of question ? Thank you.  
  > **username_1**: My intuition is that if you have the partition function and existing Ising data there would be an algorithms for seeding a Deep Neural network in an ideal way so as to allow for learning the interaction coefficients in the network much faster. Knowing the function a DNN is attempting to learn/emulate allows for the resources of a NN to be used in a more optimal, by elimination of unnecessary variables/features. I'll keep my eye out for such research.  
  
  ---
  
  