Body: <p>I am developing a particular implementation of VAE, and, how usually one does while implementing any architecture, I passed a random input to the model to test if everything worked fine (e.g. check for size mismatches, device shenanigans, etc..). While I usually just check that input, intermediate and final shapes are correct, this time I decided to also visualize the ouput on that random input, not expecting much more than pure noise.</p>
  <p><a href="https://i.sstatic.net/etCmr.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/etCmr.jpg" alt="" /></a></p>
  <p>And while of course it isn't anything semantically significant (especially since the input itself has no information, technically), <strong>it also isn't complete garbage</strong>, presenting itself way less random than the input, and clearly <strong>having regions where a certain tone prevails on others</strong>.</p>
  <p>Out of curiosity, I also tested just the decoder part of the network, by passing a normal latent sample.</p>
  <p><a href="https://i.sstatic.net/K0bNl.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/K0bNl.png" alt="" /></a></p>
  <p>It seems clear that the main culprit of this curious behaviour is the decoder network, which would lead me to believe that something similar may also happen with decoder based models such as GAN's.</p>
  <p>Now this leaves me with a burning desire to understand: <strong>why?</strong></p>
  <p>Of course, we don't really have any guarantees on the distribution of the output of a random network (or at least I think), and the decoder I am using is also a pretty complicated one, so God knows what those millions of random parameters are doing, but at the same time, I feel like I cannot be the first person to have wondered what kind of patterns are expected, if any.</p>
  <p>The only reason I can think of that could <strong>maybe</strong> explain this behaviour, and I don't really have much to prove it, is that since there are so many inputs and the parameters to the network, it is easy to hit some combinations of <code>layer_input/layer_weight</code> for which some kind of cascade effect triggers, which in turn produces these regions.</p>
  <p>I know this may be too much of an open ended question, but at the same time, I feel like it is too interesting not to ask, especially since I am not looking for answers specifically related to my model architecture but rather for a more general understanding of this phenomena. <strong>Is this behaviour expected? Could this be indicative of a good/bad model architecture or weight initialization? Is this just my primate brain looking for patterns where there really aren't any?</strong></p>
  <p>Thank you so much in advance for any responses!</p>
  <p><strong>EDIT</strong>: I also tried on a non-random input (FFHQ-256 for those wondering), and in the output you can actually see that, inspite the network being untrained, <strong>one can make out some faint outlines of the face!</strong></p>
  <p><a href="https://i.sstatic.net/zVc3p.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/zVc3p.png" alt="" /></a>
  <a href="https://i.sstatic.net/SiheF.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/SiheF.png" alt="enter image description here" /></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-19T04:34:43.283Z
Comments.Text: For your first input/output diagram have you visualized and compared with several trials with same seeded random input but different encoder/decoder randomly initialized weights, having similar patterns in the VAE output? For the bottom diagrams since the latent are conditioned on input, sure there'll be footprint in the output even without any training.
Comments.UserDisplayName: cinch
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-19T08:21:45.800Z
Comments.Text: @cinch It the output patterns **positions** and **color** wildly vary depending of the network initialization. I also tried different weight initialization strategies, such as xavier, kaiming, normal, orthogonal, and that varies the **pattern types**. I will probably add these findings in the post later, as for now I am at uni :D
Comments.UserDisplayName: GPU'njoyer
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-19T13:26:19.357Z
Comments.Text: Do you have linked or shared weights between encoder and decoder networks? For autoencoders that is usually an optional setting (and it may default to true in the library you are using). If so, that could go some way to explaining visible influence in the untrained network. There may be other architecture choices that impact this, so maybe worth sharing details of the model you are working with
Comments.UserDisplayName: Neil Slater
Tags: <generative-model><variational-autoencoder><image-generation><architecture>
Title: How random should an untrained generative AI output really be?
UserDisplayName: GPU'njoyer
ThreadText: # How random should an untrained generative AI output really be?
  
  (Asked by: username_0 on 2024-03-18)
  
  I am developing a particular implementation of VAE, and, how usually one does while implementing any architecture, I passed a random input to the model to test if everything worked fine (e.g. check for size mismatches, device shenanigans, etc..). While I usually just check that input, intermediate and final shapes are correct, this time I decided to also visualize the ouput on that random input, not expecting much more than pure noise.
  
  [![](https://i.sstatic.net/etCmr.jpg)](https://i.sstatic.net/etCmr.jpg)
  
  And while of course it isn't anything semantically significant (especially since the input itself has no information, technically), **it also isn't complete garbage**, presenting itself way less random than the input, and clearly **having regions where a certain tone prevails on others**.
  
  Out of curiosity, I also tested just the decoder part of the network, by passing a normal latent sample.
  
  [![](https://i.sstatic.net/K0bNl.png)](https://i.sstatic.net/K0bNl.png)
  
  It seems clear that the main culprit of this curious behaviour is the decoder network, which would lead me to believe that something similar may also happen with decoder based models such as GAN's.
  
  Now this leaves me with a burning desire to understand: **why?**
  
  Of course, we don't really have any guarantees on the distribution of the output of a random network (or at least I think), and the decoder I am using is also a pretty complicated one, so God knows what those millions of random parameters are doing, but at the same time, I feel like I cannot be the first person to have wondered what kind of patterns are expected, if any.
  
  The only reason I can think of that could **maybe** explain this behaviour, and I don't really have much to prove it, is that since there are so many inputs and the parameters to the network, it is easy to hit some combinations of `layer_input/layer_weight` for which some kind of cascade effect triggers, which in turn produces these regions.
  
  I know this may be too much of an open ended question, but at the same time, I feel like it is too interesting not to ask, especially since I am not looking for answers specifically related to my model architecture but rather for a more general understanding of this phenomena. **Is this behaviour expected? Could this be indicative of a good/bad model architecture or weight initialization? Is this just my primate brain looking for patterns where there really aren't any?**
  
  Thank you so much in advance for any responses!
  
  **EDIT**: I also tried on a non-random input (FFHQ-256 for those wondering), and in the output you can actually see that, inspite the network being untrained, **one can make out some faint outlines of the face!**
  
  [![](https://i.sstatic.net/zVc3p.png)](https://i.sstatic.net/zVc3p.png)
  [![enter image description here](https://i.sstatic.net/SiheF.png)](https://i.sstatic.net/SiheF.png)
  
  **Tags:** generative-model, variational-autoencoder, image-generation, architecture
  
  > **Comments:**
  > **username_1**: For your first input/output diagram have you visualized and compared with several trials with same seeded random input but different encoder/decoder randomly initialized weights, having similar patterns in the VAE output? For the bottom diagrams since the latent are conditioned on input, sure there'll be footprint in the output even without any training.  
  > **username_0**: @username_1 It the output patterns **positions** and **color** wildly vary depending of the network initialization. I also tried different weight initialization strategies, such as xavier, kaiming, normal, orthogonal, and that varies the **pattern types**. I will probably add these findings in the post later, as for now I am at uni :D  
  > **username_2**: Do you have linked or shared weights between encoder and decoder networks? For autoencoders that is usually an optional setting (and it may default to true in the library you are using). If so, that could go some way to explaining visible influence in the untrained network. There may be other architecture choices that impact this, so maybe worth sharing details of the model you are working with  
  
  ---
  
  ## 0 answers
  
  