Answers.Body: <p>Indeed you're intuition is correct, a Boltzmann machine is able to hold more than a Hopfield network in its memory because of its stochastic nature <a href="http://ieeexplore.ieee.org/document/487813/" rel="nofollow noreferrer">as explored in this paper.</a> In the paper they note that the capacity is around 0.6. After this ratio it starts to break down and adds much more noise to the recalled patterns.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-10-15T03:19:55.693Z
Answers.LastActivityDate: 2017-10-15T03:19:55.693Z
Answers.UserDisplayName: Jaden Travnik
Body: <p><sub>This is from a closed beta for AI, with this question being posted by user number 47. All credit to them. </sub></p>
  
  <hr>
  
  <p>According to <a href="https://en.wikipedia.org/wiki/Boltzmann_machine" rel="noreferrer">Wikipedia</a>,</p>
  
  <blockquote>
    <p>Boltzmann machines can be seen as the stochastic, generative counterpart of Hopfield nets.</p>
  </blockquote>
  
  <p>Both are recurrent neural networks that can be trained to learn of bit patterns. Then when presented with a partial pattern, the net will retrieve the full complete pattern.</p>
  
  <p>Hopfield networks have been proven to have a capacity of 0.138 (e.g. approximately 138 bit vectors can be recalled from storage for every 1000 nodes, Hertz 1991).</p>
  
  <p>As a Boltzmann machine is stochastic, my understanding is that it would not necessarily always show the same pattern when the energy difference between one stored pattern and another is similar. But because of this stochasticity, maybe it allows for denser pattern storage but without the guarantee that you'll always get the "closest" pattern in terms of energy difference. Would this be true? Or would a Hopfield net be able to store more patterns?</p>
  
Tags: <neural-networks><comparison><recurrent-neural-networks>
Title: Could a Boltzmann machine store more patterns than a Hopfield net?
UserDisplayName: Mithical
ThreadText: # Could a Boltzmann machine store more patterns than a Hopfield net?
  
  (Asked by: username_0 on 2016-08-10)
  
  This is from a closed beta for AI, with this question being posted by user number 47. All credit to them.
  
  ---
  
  According to [Wikipedia](https://en.wikipedia.org/wiki/Boltzmann_machine),
  
  > Boltzmann machines can be seen as the stochastic, generative counterpart of Hopfield nets.
  
  Both are recurrent neural networks that can be trained to learn of bit patterns. Then when presented with a partial pattern, the net will retrieve the full complete pattern.
  
  Hopfield networks have been proven to have a capacity of 0.138 (e.g. approximately 138 bit vectors can be recalled from storage for every 1000 nodes, Hertz 1991).
  
  As a Boltzmann machine is stochastic, my understanding is that it would not necessarily always show the same pattern when the energy difference between one stored pattern and another is similar. But because of this stochasticity, maybe it allows for denser pattern storage but without the guarantee that you'll always get the "closest" pattern in terms of energy difference. Would this be true? Or would a Hopfield net be able to store more patterns?
  
  **Tags:** neural-networks, comparison, recurrent-neural-networks
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2017-10-15. Score: 4
  
  Indeed you're intuition is correct, a Boltzmann machine is able to hold more than a Hopfield network in its memory because of its stochastic nature [as explored in this paper.](http://ieeexplore.ieee.org/document/487813/) In the paper they note that the capacity is around 0.6. After this ratio it starts to break down and adds much more noise to the recalled patterns.
  
  
  ---
  
  