Body: <p>Seung et.al recently published <a href="http://arxiv.org/abs/2005.12126" rel="nofollow noreferrer">GameGAN</a> paper, GameGAN learned and stored the whole Pacman game and was able to reproduce it without a game engine. The uniqueness of GameGAN is that it had added memory to its discriminator/generator which helped it to store the game states.</p>
  
  <p><a href="https://i.sstatic.net/iOb2m.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/iOb2m.png" alt="enter image description here"></a></p>
  
  <p>In Bayesian interpretation, a supervised learning system learns by optimizing weights which maximizes a likelihood function.</p>
  
  <p><span class="math-container">$$\hat{\boldsymbol{\theta}} = \mathop{\mathrm{argmax}} _ {\boldsymbol{\theta}} P(X \mid \boldsymbol{\theta})$$</span></p>
  
  <p>Will adding memory which can store prior information makes GameGAN a Bayesian learning system?</p>
  
  <p>Can GameGAN or similar neural network with memory can be considered as a bayesian learning system. If yes, then which of these two equations(or something other) correctly explains this system (considering prior as memory)?</p>
  
  <ol>
  <li><span class="math-container">$$\mathop{\mathrm{argmax}} \frac{P(X \mid \boldsymbol{\theta})P(\boldsymbol{\theta})}{P(X)}$$</span></li>
  </ol>
  
  <p>or</p>
  
  <ol start="2">
  <li><span class="math-container">$$\mathop{\mathrm{argmax}} \frac{P(X_t \mid \boldsymbol{X^{t+1}})P(\boldsymbol{X^{t+1}})}{P(X_t)}$$</span></li>
  </ol>
  
  <p>PS: I understand GAN's are unsupervised learning systems, but we can assume discriminator and generator models separately trying to find weights that maximize their individual likelihood function.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-13T13:52:21.780Z
Comments.Text: If you're maximizing the likelihood (maximum likelihood estimation), that doesn't make the system Bayesian, in the sense that you just get a point estimate rather than a distribution and you are not specifying any prior. In Bayesian learning, you need to be able to specify a prior on the parameters and you often get a posterior distribution over them. So, your sentence "In Bayesian interpretation, a supervised learning system learns by optimizing weights which maximizes a likelihood function" is wrong, in the sense that's just a **probabilistic** interpretation (rather than a Bayesian one).
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-17T12:17:37.757Z
Comments.Text: You might also be interested in [this post](https://ai.stackexchange.com/q/17670/2444).
Comments.UserDisplayName: nbro
Tags: <generative-adversarial-networks><bayesian-deep-learning><bayesian-statistics><game-gan>
Title: Will adding memory to a supervised learning system makes it into a Bayesian learning system?
UserDisplayName: Eka
ThreadText: # Will adding memory to a supervised learning system makes it into a Bayesian learning system?
  
  (Asked by: username_0 on 2020-06-13)
  
  Seung et.al recently published [GameGAN](http://arxiv.org/abs/2005.12126) paper, GameGAN learned and stored the whole Pacman game and was able to reproduce it without a game engine. The uniqueness of GameGAN is that it had added memory to its discriminator/generator which helped it to store the game states.
  
  [![enter image description here](https://i.sstatic.net/iOb2m.png)](https://i.sstatic.net/iOb2m.png)
  
  In Bayesian interpretation, a supervised learning system learns by optimizing weights which maximizes a likelihood function.
  
  $$\hat{\boldsymbol{\theta}} = \mathop{\mathrm{argmax}} \_ {\boldsymbol{\theta}} P(X \mid \boldsymbol{\theta})$$
  
  Will adding memory which can store prior information makes GameGAN a Bayesian learning system?
  
  Can GameGAN or similar neural network with memory can be considered as a bayesian learning system. If yes, then which of these two equations(or something other) correctly explains this system (considering prior as memory)?
  
  1. $$\mathop{\mathrm{argmax}} \frac{P(X \mid \boldsymbol{\theta})P(\boldsymbol{\theta})}{P(X)}$$
  
  or
  
  2. $$\mathop{\mathrm{argmax}} \frac{P(X\_t \mid \boldsymbol{X^{t+1}})P(\boldsymbol{X^{t+1}})}{P(X\_t)}$$
  
  PS: I understand GAN's are unsupervised learning systems, but we can assume discriminator and generator models separately trying to find weights that maximize their individual likelihood function.
  
  **Tags:** generative-adversarial-networks, bayesian-deep-learning, bayesian-statistics, game-gan
  
  > **Comments:**
  > **username_1**: If you're maximizing the likelihood (maximum likelihood estimation), that doesn't make the system Bayesian, in the sense that you just get a point estimate rather than a distribution and you are not specifying any prior. In Bayesian learning, you need to be able to specify a prior on the parameters and you often get a posterior distribution over them. So, your sentence "In Bayesian interpretation, a supervised learning system learns by optimizing weights which maximizes a likelihood function" is wrong, in the sense that's just a **probabilistic** interpretation (rather than a Bayesian one).  
  > **username_1**: You might also be interested in [this post](https://ai.stackexchange.com/q/17670/2444).  
  
  ---
  
  ## 0 answers
  
  