Body: <p>I made an autoencoder to, ideally, turn an image into seemingly random numbers(Using a loss that determines randomness) and turn those random numbers into the original image. The results were kind of meh. The ideal encoded image would just look like rainbow static, but I got this instead:</p>
  <p><a href="https://i.sstatic.net/SvwP9.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/SvwP9.png" alt="Badly Encoded Image" /></a></p>
  <p>The decoded image was also kind of bad, but that's the same problem as the encoder. I was wondering if there was a better way to handle the image -&gt; random -&gt; image conversions better. Like maybe changing the stride length or filter size could help it learn global features better. Here is what my current encoder and decoder look like:</p>
  <pre><code>input_layer = Input(shape=input_shape)
  x = Conv2D(4, (3, 3), activation='relu', padding='same')(input_layer)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  decoded = Conv2D(3, (3, 3), activation='sigmoid', padding='same')(x)
  </code></pre>
  <p>Pls don't judge my autoencoder    Thanks</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-12-30T02:32:03.457Z
Comments.Text: Why should you be able to reconstruct the image from random values?
Comments.UserDisplayName: Dave
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-12-30T10:21:14.397Z
Comments.Text: @Dave The encoded images wouldn't really be random, but I have a loss on the encoder based on how random the distribution of images is. This way, it can create a random-looking image for each real image while being able to sneak information through to the decoder. I just have a problem where CNNs are good at spatially related images, but not random numbers.
Comments.UserDisplayName: Nathanael Suarez
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-12-30T10:33:34.487Z
Comments.Text: That last part makes it sound like the CNN is doing what it’s supposed to do. It isn’t supposed to invent patterns out of randomness (that’s overfitting, at least a little loosely speaking).
Comments.UserDisplayName: Dave
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-12-30T18:05:59.613Z
Comments.Text: @NathanaelSuarez To receive an answer quickly, it may help to clearly outline your question in the post.
Comments.UserDisplayName: DeepQZero
Tags: <convolutional-neural-networks><autoencoders><contrastive-learning>
Title: Is there a better way for my CNN to handle random values?
UserDisplayName: Nathanael Suarez
ThreadText: # Is there a better way for my CNN to handle random values?
  
  (Asked by: username_0 on 2023-12-30)
  
  I made an autoencoder to, ideally, turn an image into seemingly random numbers(Using a loss that determines randomness) and turn those random numbers into the original image. The results were kind of meh. The ideal encoded image would just look like rainbow static, but I got this instead:
  
  The decoded image was also kind of bad, but that's the same problem as the encoder. I was wondering if there was a better way to handle the image -> random -> image conversions better. Like maybe changing the stride length or filter size could help it learn global features better. Here is what my current encoder and decoder look like:
  input_layer = Input(shape=input_shape)
  x = Conv2D(4, (3, 3), activation='relu', padding='same')(input_layer)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  x = Conv2D(4, (3, 3), activation='relu', padding='same', groups=1)(x)
  decoded = Conv2D(3, (3, 3), activation='sigmoid', padding='same')(x)
  
  Pls don't judge my autoencoder    Thanks
  
  **Tags:** convolutional-neural-networks, autoencoders, contrastive-learning
  > **Comments:**
  > **username_1**: Why should you be able to reconstruct the image from random values?  
  > **username_0**: @Dave The encoded images wouldn't really be random, but I have a loss on the encoder based on how random the distribution of images is. This way, it can create a random-looking image for each real image while being able to sneak information through to the decoder. I just have a problem where CNNs are good at spatially related images, but not random numbers.  
  > **username_1**: That last part makes it sound like the CNN is doing what it’s supposed to do. It isn’t supposed to invent patterns out of randomness (that’s overfitting, at least a little loosely speaking).  
  > **username_2**: @NathanaelSuarez To receive an answer quickly, it may help to clearly outline your question in the post.  
  
  ---
  
  ## 0 answers
  
  