Body: <p>I would like to use the bottleneck layer of U-Net (the last layer of the encoder) to calculate the similarity between two images. For that, I have to somehow flatten the last layer of the encoder. In my opinion, there are two approaches:</p>
  
  <ol>
  <li><p>Take the last layer which in my case is <span class="math-container">$4 \times 4 \times 16$</span> and flatten it to 1D</p></li>
  <li><p>Add a dense before the decoder and then reshape the dense 1D layer into 3D</p></li>
  </ol>
  
  <p>For the second case, I am not sure how this would affect the network. Arbitrarily reshaping a 1D array into a 3D tensor. Could that introduce weird artifacts? Does someone have experience in a similar problem?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-06-19T04:48:56.727Z
Comments.Text: If I understood you correctly it's the same (architecturally). Decoder will reshape dense layer into 3d tensor anyway in U-net
Comments.UserDisplayName: mirror2image
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-06-19T22:33:43.777Z
Comments.Text: I don't think that's true. In the paper the bottleneck layer is of dimension 30 x 30 x 1024.
Comments.UserDisplayName: oezguensi
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-06-20T04:50:28.553Z
Comments.Text: Anyway shouldn't be a problem. Dense layer is just 1x1xN convolution layer and however you down/upsample it shouldn't change behavior too much.
Comments.UserDisplayName: mirror2image
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-06-20T20:18:31.097Z
Comments.Text: I think it only doesn't change things if - when shaped into 3D again - the same pixels  end up in the same spot. Otherwise the spatial information gets lost.
Comments.UserDisplayName: oezguensi
Tags: <convolutional-neural-networks><unsupervised-learning><u-net>
Title: How can I use the bottleneck layer of the U-net to calculate the similarity between two images?
UserDisplayName: oezguensi
ThreadText: # How can I use the bottleneck layer of the U-net to calculate the similarity between two images?
  
  (Asked by: username_0 on 2019-06-19)
  
  I would like to use the bottleneck layer of U-Net (the last layer of the encoder) to calculate the similarity between two images. For that, I have to somehow flatten the last layer of the encoder. In my opinion, there are two approaches:
  
  1. Take the last layer which in my case is $4 \times 4 \times 16$ and flatten it to 1D
  2. Add a dense before the decoder and then reshape the dense 1D layer into 3D
  
  For the second case, I am not sure how this would affect the network. Arbitrarily reshaping a 1D array into a 3D tensor. Could that introduce weird artifacts? Does someone have experience in a similar problem?
  
  **Tags:** convolutional-neural-networks, unsupervised-learning, u-net
  
  > **Comments:**
  > **username_1**: If I understood you correctly it's the same (architecturally). Decoder will reshape dense layer into 3d tensor anyway in U-net  
  > **username_0**: I don't think that's true. In the paper the bottleneck layer is of dimension 30 x 30 x 1024.  
  > **username_1**: Anyway shouldn't be a problem. Dense layer is just 1x1xN convolution layer and however you down/upsample it shouldn't change behavior too much.  
  > **username_0**: I think it only doesn't change things if - when shaped into 3D again - the same pixels  end up in the same spot. Otherwise the spatial information gets lost.  
  
  ---
  
  ## 0 answers
  
  