Answers.Body: <p>You are missing a <code>Reshape</code> step between layers 9 and 10. In addition I suggest you to add an activation function to your dense layers.</p>
  <p><strong>Edit 1</strong>:</p>
  <p>Actually in Keras we don't usually care about the tensor's first dimension, since it is the batch size: <a href="https://keras.io/api/layers/reshaping_layers/reshape/" rel="nofollow noreferrer">docs</a>. I assume you set the <code>batch_size</code> at your <a href="https://keras.io/api/layers/core_layers/input/" rel="nofollow noreferrer">input layer</a>, so the tensors' dimensions first value isn't <code>None</code>?</p>
  <p>Check the dimensions of your <code>l7</code>, according to the attached image it should be <code>(60, 512)</code>. <code>l8</code> should be <code>Dense(512, activation = 'leaky_relu')(l7)</code> and <code>l9</code> would be <code>Dense(128*4*4*4, activation = 'leaky_relu')(l8)</code> which can be reshaped to <code>(60, 4, 4, 4, 128)</code> by calling <code>Reshape((4, 4, 4, 128))(l9)</code>.</p>
  <p>Now that I look at your implementation more carefully I noticed several issues:</p>
  <ul>
  <li>It has six Conv2D layers but the reference architecture has seven</li>
  <li>The number of filters and the kernel size doesn't match, except for the <code>l1</code> layer. For example <code>l2</code> should have 64 filters and a size of 5. And the number of filters increases as we go deeper, the last layer has 512 of them but yours has only one.</li>
  <li>The reference architecture seems to use pooling layers, their resolution is halved at each step.</li>
  </ul>
  <p>I suggest you try some simpler tutorials first, to get familiar with tensor sizes and how they relate to different network layers and their parameters.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-18T18:39:35.373Z
Answers.Text: ```#latent variable z
  
  
  l7 = Reshape((60,512))(l6)
  l8 = Dense((60), activation = 'leaky_relu')(l7)
  l9 = Dense((60), activation = 'leaky_relu')(l8)
  l10 = Reshape((60, 4, 4, 4))(l9)```
  
  i did this but its giving me error-> ```ValueError: total size of new array must be unchanged, input_shape = [60, 60], output_shape = [60, 4, 4, 4]```
Answers.UserDisplayName: arizona_3
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-27T05:26:11.277Z
Answers.Text: Thanks for your explanation, i would also like to learn that whether upsampling layers needs to be inserted in latent variable layers as well? since between dense and reshape there seems to be increase in resolution?
Answers.UserDisplayName: arizona_3
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-27T17:24:13.167Z
Answers.Text: I'm fairly sure they forgot to mention the batch size there, it is the only block which's size doesn't start with `60 x`. And also it is a bit odd that a fully connected layer would have a such high output dimension, that is why I think it is actually 8192 (= 128*4*4*4) before the reshape.
Answers.UserDisplayName: NikoNyrh
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-28T04:56:43.793Z
Answers.Text: Niko i made some changes and have updated the code further some error still persists...unable to understand where exactly should i tweak...
Answers.UserDisplayName: arizona_3
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-18T18:20:07.383Z
Answers.LastActivityDate: 2022-07-19T09:01:01.263Z
Answers.LastEditDate: 2022-07-19T09:01:01.263Z
Answers.UserDisplayName: NikoNyrh
Body: <p>I went through a research paper (&quot;<a href="https://www.mdpi.com/2227-7390/9/18/2288" rel="nofollow noreferrer">Voxel-Based 3D Object Reconstruction from Single 2D Image Using Variational Autoencoders</a>&quot;) and tried to implement the approach following this diagram:</p>
  <p>![link to image of reference network- <a href="https://ibb.co/4JgbQ9s" rel="nofollow noreferrer">https://ibb.co/4JgbQ9s</a></p>
  <p>Here is my implementation for the same:</p>
  <pre><code>image = Input(shape=(None, None, 3))
  
  # Encoder
  l1 = Conv2D(64, (3,3), strides = (2), padding='same', activation='leaky_relu')(image)  
  l2 = MaxPooling2D(padding='same')(l1)
  l3 = Conv2D(32, (5,5), strides = (2), padding='same', activation='leaky_relu')(l2)
  l4 = MaxPooling2D(padding='same')(l3)
  l5 = Conv2D(16, (7,7), strides = (2), padding='same', activation='leaky_relu')(l4)
  l6 = MaxPooling2D(padding='same')(l5)
  l7 = Conv2D(8, (5, 5), strides = (2), padding = 'same', activation = 'leaky_relu')(l6)
  l8 = MaxPooling2D(padding='same')(l7)
  l9 = Conv2D(4, (3, 3), strides = (2), padding = 'same', activation = 'leaky_relu')(l8)
  l10 = MaxPooling2D(padding='same')(l9)
  l11 = Conv2D(2, (4, 4), strides = (2), padding = 'same', activation = 'leaky_relu')(l10)
  l12 = MaxPooling2D(padding='same')(l11)
  l13 = Conv2D(1, (2, 2), strides = (2), padding = 'same', activation = 'leaky_relu')(l12)
  
  #latent variable z
  l14 = Reshape((60,512))(l13)
  l15 = Dense((60*512), activation = 'leaky_relu')(l14)
  l16 = Dense((128*4*4*4), activation = 'leaky_relu')(l15)
  l17 = Reshape((60,4,4,4,128))(l16)
  
  #Decoder
  l18 = UpSampling3D()(l17)
  l19 = Conv3DTranspose(60, (8, 8, 8), strides = (64), padding='same', activation = 'leaky_relu') (l17)
  l20 = UpSampling3D()(l19)
  l21 = Conv3DTranspose(60, (16,16,16), strides =(32), padding='same', activation = 'leaky_relu')(l20)
  l22 = UpSampling3D()(l21)
  l23 = Conv3DTranspose(60, (32, 32, 32), strides = (32), padding='same', activation = 'lealy_relu')(l22)
  l24 = UpSampling3D()(l23)
  l25 = Conv3DTranspose(60, (64, 64, 64), strides = (24), padding='same', activation = 'leaky_relu')(l24)
  l26 = UpSampling3D()(l25)
  l27 = Conv3DTranspose(60, (64, 64, 64), strides = (1), padding='same', activation = 'leaky_relu')(l26)
  
  model3D = Model(image, l27)
  </code></pre>
  <p>This gives me error for l10 saying:</p>
  <pre><code>---------------------------------------------------------------------------
  ValueError                                Traceback (most recent call last)
  /tmp/ipykernel_33/351640059.py in &lt;module&gt;
       24 #Decoder
       25 l18 = UpSampling3D()(l17)
  ---&gt; 26 l19 = Conv3DTranspose(60, (8, 8, 8), strides = (64), padding='same', activation = 'leaky_relu') (l17)
       27 l20 = UpSampling3D()(l19)
       28 l21 = Conv3DTranspose(60, (16,16,16), strides =(32), padding='same', activation = 'leaky_relu')(l20)
  
  /opt/conda/lib/python3.7/site-packages/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
      975     if _in_functional_construction_mode(self, inputs, args, kwargs, input_list):
      976       return self._functional_construction_call(inputs, args, kwargs,
  --&gt; 977                                                 input_list)
      978 
      979     # Maintains info about the `Layer.call` stack.
  
  /opt/conda/lib/python3.7/site-packages/keras/engine/base_layer.py in _functional_construction_call(self, inputs, args, kwargs, input_list)
     1113       # Check input assumptions set after layer building, e.g. input shape.
     1114       outputs = self._keras_tensor_symbolic_call(
  -&gt; 1115           inputs, input_masks, args, kwargs)
     1116 
     1117       if outputs is None:
  
  /opt/conda/lib/python3.7/site-packages/keras/engine/base_layer.py in _keras_tensor_symbolic_call(self, inputs, input_masks, args, kwargs)
      846       return tf.nest.map_structure(keras_tensor.KerasTensor, output_signature)
      847     else:
  --&gt; 848       return self._infer_output_signature(inputs, args, kwargs, input_masks)
      849 
      850   def _infer_output_signature(self, inputs, args, kwargs, input_masks):
  
  /opt/conda/lib/python3.7/site-packages/keras/engine/base_layer.py in _infer_output_signature(self, inputs, args, kwargs, input_masks)
      884           # overridden).
      885           # TODO(kaftan): do we maybe_build here, or have we already done it?
  --&gt; 886           self._maybe_build(inputs)
      887           inputs = self._maybe_cast_inputs(inputs)
      888           outputs = call_fn(inputs, *args, **kwargs)
  
  /opt/conda/lib/python3.7/site-packages/keras/engine/base_layer.py in _maybe_build(self, inputs)
     2657         # operations.
     2658         with tf_utils.maybe_init_scope(self):
  -&gt; 2659           self.build(input_shapes)  # pylint:disable=not-callable
     2660       # We must set also ensure that the layer is marked as built, and the build
     2661       # shape is stored since user defined build functions may not be calling
  
  /opt/conda/lib/python3.7/site-packages/keras/layers/convolutional.py in build(self, input_shape)
     1546     if len(input_shape) != 5:
     1547       raise ValueError('Inputs should have rank 5, received input shape:',
  -&gt; 1548                        str(input_shape))
     1549     channel_axis = self._get_channel_axis()
     1550     if input_shape.dims[channel_axis].value is None:
  
  ValueError: ('Inputs should have rank 5, received input shape:', '(None, 60, 4, 4, 4, 128)')&quot;```
  
  Any help and guidance is appreciated.
  
  </code></pre>
  
Tags: <machine-learning><computer-vision><autoencoders><3d-convolution><2d-convolution>
Title: Reconstructing 3D models from 2D images using autoencoders
UserDisplayName: arizona_3
ThreadText: # Reconstructing 3D models from 2D images using autoencoders
  
  (Asked by: username_0 on 2022-07-18)
  
  I went through a research paper ("Voxel-Based 3D Object Reconstruction from Single 2D Image Using Variational Autoencoders") and tried to implement the approach following this diagram:
  ![link to image of reference network- https://ibb.co/4JgbQ9s
  Here is my implementation for the same:
  image = Input(shape=(None, None, 3))
  
  # Encoder
  l1 = Conv2D(64, (3,3), strides = (2), padding='same', activation='leaky_relu')(image)  
  l2 = MaxPooling2D(padding='same')(l1)
  l3 = Conv2D(32, (5,5), strides = (2), padding='same', activation='leaky_relu')(l2)
  l4 = MaxPooling2D(padding='same')(l3)
  l5 = Conv2D(16, (7,7), strides = (2), padding='same', activation='leaky_relu')(l4)
  l6 = MaxPooling2D(padding='same')(l5)
  l7 = Conv2D(8, (5, 5), strides = (2), padding = 'same', activation = 'leaky_relu')(l6)
  l8 = MaxPooling2D(padding='same')(l7)
  l9 = Conv2D(4, (3, 3), strides = (2), padding = 'same', activation = 'leaky_relu')(l8)
  l10 = MaxPooling2D(padding='same')(l9)
  l11 = Conv2D(2, (4, 4), strides = (2), padding = 'same', activation = 'leaky_relu')(l10)
  l12 = MaxPooling2D(padding='same')(l11)
  l13 = Conv2D(1, (2, 2), strides = (2), padding = 'same', activation = 'leaky_relu')(l12)
  
  #latent variable z
  l14 = Reshape((60,512))(l13)
  l15 = Dense((60*512), activation = 'leaky_relu')(l14)
  l16 = Dense((128*4*4*4), activation = 'leaky_relu')(l15)
  l17 = Reshape((60,4,4,4,128))(l16)
  
  #Decoder
  l18 = UpSampling3D()(l17)
  l19 = Conv3DTranspose(60, (8, 8, 8), strides = (64), padding='same', activation = 'leaky_relu') (l17)
  l20 = UpSampling3D()(l19)
  l21 = Conv3DTranspose(60, (16,16,16), strides =(32), padding='same', activation = 'leaky_relu')(l20)
  l22 = UpSampling3D()(l21)
  l23 = Conv3DTranspose(60, (32, 32, 32), strides = (32), padding='same', activation = 'lealy_relu')(l22)
  l24 = UpSampling3D()(l23)
  l25 = Conv3DTranspose(60, (64, 64, 64), strides = (24), padding='same', activation = 'leaky_relu')(l24)
  l26 = UpSampling3D()(l25)
  l27 = Conv3DTranspose(60, (64, 64, 64), strides = (1), padding='same', activation = 'leaky_relu')(l26)
  
  model3D = Model(image, l27)
  
  This gives me error for l10 saying:
  ---------------------------------------------------------------------------
  ValueError                                Traceback (most recent call last)
  /tmp/ipykernel_33/351640059.py in 
       24 #Decoder
       25 l18 = UpSampling3D()(l17)
  ---> 26 l19 = Conv3DTranspose(60, (8, 8, 8), strides = (64), padding='same', activation = 'leaky_relu') (l17)
       27 l20 = UpSampling3D()(l19)
       28 l21 = Conv3DTranspose(60, (16,16,16), strides =(32), padding='same', activation = 'leaky_relu')(l20)
  
  /opt/conda/lib/python3.7/site-packages/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
      975     if _in_functional_construction_mode(self, inputs, args, kwargs, input_list):
      976       return self._functional_construction_call(inputs, args, kwargs,
  --> 977                                                 input_list)
      978 
      979     # Maintains info about the `Layer.call` stack.
  
  /opt/conda/lib/python3.7/site-packages/keras/engine/base_layer.py in _functional_construction_call(self, inputs, args, kwargs, input_list)
     1113       # Check input assumptions set after layer building, e.g. input shape.
     1114       outputs = self._keras_tensor_symbolic_call(
  -> 1115           inputs, input_masks, args, kwargs)
     1116 
     1117       if outputs is None:
  
  /opt/conda/lib/python3.7/site-packages/keras/engine/base_layer.py in _keras_tensor_symbolic_call(self, inputs, input_masks, args, kwargs)
      846       return tf.nest.map_structure(keras_tensor.KerasTensor, output_signature)
      847     else:
  --> 848       return self._infer_output_signature(inputs, args, kwargs, input_masks)
      849 
      850   def _infer_output_signature(self, inputs, args, kwargs, input_masks):
  
  /opt/conda/lib/python3.7/site-packages/keras/engine/base_layer.py in _infer_output_signature(self, inputs, args, kwargs, input_masks)
      884           # overridden).
      885           # TODO(kaftan): do we maybe_build here, or have we already done it?
  --> 886           self._maybe_build(inputs)
      887           inputs = self._maybe_cast_inputs(inputs)
      888           outputs = call_fn(inputs, *args, **kwargs)
  
  /opt/conda/lib/python3.7/site-packages/keras/engine/base_layer.py in _maybe_build(self, inputs)
     2657         # operations.
     2658         with tf_utils.maybe_init_scope(self):
  -> 2659           self.build(input_shapes)  # pylint:disable=not-callable
     2660       # We must set also ensure that the layer is marked as built, and the build
     2661       # shape is stored since user defined build functions may not be calling
  
  /opt/conda/lib/python3.7/site-packages/keras/layers/convolutional.py in build(self, input_shape)
     1546     if len(input_shape) != 5:
     1547       raise ValueError('Inputs should have rank 5, received input shape:',
  -> 1548                        str(input_shape))
     1549     channel_axis = self._get_channel_axis()
     1550     if input_shape.dims[channel_axis].value is None:
  
  ValueError: ('Inputs should have rank 5, received input shape:', '(None, 60, 4, 4, 4, 128)')"```
  
  Any help and guidance is appreciated.
  
  **Tags:** machine-learning, computer-vision, autoencoders, 3d-convolution, 2d-convolution
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-07-18. Score: 0
  
  You are missing a `Reshape` step between layers 9 and 10. In addition I suggest you to add an activation function to your dense layers.
  
  **Edit 1**:
  
  Actually in Keras we don't usually care about the tensor's first dimension, since it is the batch size: [docs](https://keras.io/api/layers/reshaping_layers/reshape/). I assume you set the `batch_size` at your [input layer](https://keras.io/api/layers/core_layers/input/), so the tensors' dimensions first value isn't `None`?
  
  Check the dimensions of your `l7`, according to the attached image it should be `(60, 512)`. `l8` should be `Dense(512, activation = 'leaky_relu')(l7)` and `l9` would be `Dense(128*4*4*4, activation = 'leaky_relu')(l8)` which can be reshaped to `(60, 4, 4, 4, 128)` by calling `Reshape((4, 4, 4, 128))(l9)`.
  
  Now that I look at your implementation more carefully I noticed several issues:
  
  * It has six Conv2D layers but the reference architecture has seven
  * The number of filters and the kernel size doesn't match, except for the `l1` layer. For example `l2` should have 64 filters and a size of 5. And the number of filters increases as we go deeper, the last layer has 512 of them but yours has only one.
  * The reference architecture seems to use pooling layers, their resolution is halved at each step.
  
  I suggest you try some simpler tutorials first, to get familiar with tensor sizes and how they relate to different network layers and their parameters.
  
  > **Comments:**
  > **username_0**: ```#latent variable z   l7 = Reshape((60,512))(l6) l8 = Dense((60), activation = 'leaky_relu')(l7) l9 = Dense((60), activation = 'leaky_relu')(l8) l10 = Reshape((60, 4, 4, 4))(l9)```  i did this but its giving me error-> ```ValueError: total size of new array must be unchanged, input_shape = [60, 60], output_shape = [60, 4, 4, 4]```  
  > **username_0**: Thanks for your explanation, i would also like to learn that whether upsampling layers needs to be inserted in latent variable layers as well? since between dense and reshape there seems to be increase in resolution?  
  > **username_1**: I'm fairly sure they forgot to mention the batch size there, it is the only block which's size doesn't start with `60 x`. And also it is a bit odd that a fully connected layer would have a such high output dimension, that is why I think it is actually 8192 (= 128*4*4*4) before the reshape.  
  > **username_0**: Niko i made some changes and have updated the code further some error still persists...unable to understand where exactly should i tweak...  
  
  ---
  
  