Answers.Body: <p>It's easier to understand in the context of the lecture. I'm assuming you're referencing this slide in <a href="http://cs231n.stanford.edu/2021/slides/2021/lecture_12.pdf" rel="nofollow noreferrer">Lecture 12</a>:
  <a href="https://i.sstatic.net/PJiWN.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/PJiWN.png" alt="CS231n Lec 12, slide 16: Taxonomy of Generative Modles: http://cs231n.stanford.edu/2021/slides/2021/lecture_12.pdf" /></a></p>
  <p>What distinguishes the FVBNs from the other explicit density models are that they have a <em>tractable density</em>: that is, that you can exactly calculate the probability of samples of your dataset (or at least, that's the assumption made by the model). <strong>This is what's meant by &quot;fully visible&quot;</strong>.</p>
  <p>Take PixelRNN/CNN, for example. Here, you model images autoregressively, where a given pixel only depends on previous pixels. This allows you to decompose the distribution over many pixels using the chain rule. <a href="http://cs231n.stanford.edu/2021/slides/2021/lecture_12.pdf" rel="nofollow noreferrer">Look at slide 23 in the lecture for more info.</a> <strong>The main idea is that this decomposition allows you to calculate the probability of a sample exactly, making it &quot;fully visible&quot;.</strong></p>
  <p>The connection to deep belief networks is a bit more iffy in my opinion. The <a href="https://www.jmlr.org/papers/volume17/16-272/16-272.pdf" rel="nofollow noreferrer">NADE paper</a> explains how the NADE forward pass is similar to an approximation of a RBM. I'm unsure if this extends to other models, however.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-04T00:19:32.647Z
Answers.LastActivityDate: 2023-09-09T05:40:49.047Z
Answers.LastEditDate: 2023-09-09T05:40:49.047Z
Answers.UserDisplayName: Alexander Wan
Body: <p>In CS231n, I saw the terminology &quot;fully visible belief network&quot;, one category of explicit density generative model such as PixelCNN and NADE.</p>
  <p>Although I can understand what this terminology wants to say, but I need more explanations in detail:</p>
  <ol>
  <li>What exactly does &quot;fully visible&quot; (or &quot;visible&quot;) mean?</li>
  <li>I have searched about deep belief net, and I believe that it is just a model in category of deep neural network but trained in unsupervised way with energy function. Am I right?</li>
  </ol>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-09-01T23:19:29.107Z
Comments.Text: deep belief net has little to do with neural network, if not sharing the concept of "neurons" but they are defined in 2 very different ways, though, i have no idea what they mean with _fully visible belief network_... maybe having no "hidden/latent variables", but it's a pretty loose definition
Comments.UserDisplayName: Alberto
Tags: <generative-model>
Title: What does "fully visible belief network" stand for?
UserDisplayName: Chris
ThreadText: # What does "fully visible belief network" stand for?
  
  (Asked by: username_0 on 2023-08-30)
  
  In CS231n, I saw the terminology "fully visible belief network", one category of explicit density generative model such as PixelCNN and NADE.
  
  Although I can understand what this terminology wants to say, but I need more explanations in detail:
  
  1. What exactly does "fully visible" (or "visible") mean?
  2. I have searched about deep belief net, and I believe that it is just a model in category of deep neural network but trained in unsupervised way with energy function. Am I right?
  
  **Tags:** generative-model
  
  > **Comments:**
  > **username_1**: deep belief net has little to do with neural network, if not sharing the concept of "neurons" but they are defined in 2 very different ways, though, i have no idea what they mean with _fully visible belief network_... maybe having no "hidden/latent variables", but it's a pretty loose definition  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-09-04. Score: 1
  
  It's easier to understand in the context of the lecture. I'm assuming you're referencing this slide in [Lecture 12](http://cs231n.stanford.edu/2021/slides/2021/lecture_12.pdf):
  [![CS231n Lec 12, slide 16: Taxonomy of Generative Modles: http://cs231n.stanford.edu/2021/slides/2021/lecture_12.pdf](https://i.sstatic.net/PJiWN.png)](https://i.sstatic.net/PJiWN.png)
  
  What distinguishes the FVBNs from the other explicit density models are that they have a *tractable density*: that is, that you can exactly calculate the probability of samples of your dataset (or at least, that's the assumption made by the model). **This is what's meant by "fully visible"**.
  
  Take PixelRNN/CNN, for example. Here, you model images autoregressively, where a given pixel only depends on previous pixels. This allows you to decompose the distribution over many pixels using the chain rule. [Look at slide 23 in the lecture for more info.](http://cs231n.stanford.edu/2021/slides/2021/lecture_12.pdf) **The main idea is that this decomposition allows you to calculate the probability of a sample exactly, making it "fully visible".**
  
  The connection to deep belief networks is a bit more iffy in my opinion. The [NADE paper](https://www.jmlr.org/papers/volume17/16-272/16-272.pdf) explains how the NADE forward pass is similar to an approximation of a RBM. I'm unsure if this extends to other models, however.
  
  
  ---
  
  