Answers.Body: <p>You can also think of a convolutional neural network (CNN) as an encoder, i.e. a neural network that learns a smaller representation of the input, which then acts as the feature vector (input) to a fully connected network (or another neural network). In fact, there are CNNs that can be thought of as auto-encoders (i.e. an encoder followed by a decoder): for example, the <a href="https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/" rel="nofollow noreferrer">u-net</a> can indeed be thought of as an auto-encoder.</p>
  <p>Although it is (almost) never the case that you transform the input to an extremely small feature vector (e.g. a number), even a single float-pointing number can encode a lot of information. For example, if you want to classify the object in the image into one of two classes (assuming there is only one main object in the image), then a floating-point is more than sufficient (in fact, you just need one bit to encode that information).</p>
  <p>This smaller representation (the <strong>feature</strong> vector) that is then fed to a fully connected network is learned based on the information in your given training data. In fact, CNNs are known as <strong>data-driven feature extractors</strong>.</p>
  <p>I am not aware of any theoretical guarantee that ensures that the learned representation is the best suited for your task (probably you need to look into <em>learning theory</em> to know more about this). In practice, the quality of the learned feature vector will mainly depend on your available data and the inductive bias (i.e. the assumptions that you make, which are also affected by the specific neural network architecture that you choose).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-25T16:44:12.847Z
Answers.Text: This may sound silly. In the answer, "In practice, the quality of the learned feature vector": is this vector the output of CNN right before we connect it to MLP or the final output.
Answers.UserDisplayName: Naveen Reddy Marthala
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-25T17:03:48.477Z
Answers.Text: @NaveenKumar Yes, exactly, that's what I meant.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-25T16:13:56.653Z
Answers.LastActivityDate: 2020-06-25T16:24:55.287Z
Answers.LastEditDate: 2020-06-25T16:24:55.287Z
Answers.UserDisplayName: nbro
Body: <p>Our length of feature representation decreases as we go deeper into the CNN, I mean to say that horizontal and vertical lengths decrease while depth(channels) increase. So, how will the input be preserved, since there won't be any data left at the end of the network, where we connect, to say Multi Layer Perceptrons?</p>
  
Tags: <deep-learning><convolutional-neural-networks><convolution>
Title: How will the input be preserved as we go deeper in CNN, where dimensions decrease drastically?
UserDisplayName: Naveen Reddy Marthala
ThreadText: # How will the input be preserved as we go deeper in CNN, where dimensions decrease drastically?
  
  (Asked by: username_0 on 2020-06-25)
  
  Our length of feature representation decreases as we go deeper into the CNN, I mean to say that horizontal and vertical lengths decrease while depth(channels) increase. So, how will the input be preserved, since there won't be any data left at the end of the network, where we connect, to say Multi Layer Perceptrons?
  
  **Tags:** deep-learning, convolutional-neural-networks, convolution
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-06-25. Score: 2 | Accepted answer
  
  You can also think of a convolutional neural network (CNN) as an encoder, i.e. a neural network that learns a smaller representation of the input, which then acts as the feature vector (input) to a fully connected network (or another neural network). In fact, there are CNNs that can be thought of as auto-encoders (i.e. an encoder followed by a decoder): for example, the [u-net](https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/) can indeed be thought of as an auto-encoder.
  
  Although it is (almost) never the case that you transform the input to an extremely small feature vector (e.g. a number), even a single float-pointing number can encode a lot of information. For example, if you want to classify the object in the image into one of two classes (assuming there is only one main object in the image), then a floating-point is more than sufficient (in fact, you just need one bit to encode that information).
  
  This smaller representation (the **feature** vector) that is then fed to a fully connected network is learned based on the information in your given training data. In fact, CNNs are known as **data-driven feature extractors**.
  
  I am not aware of any theoretical guarantee that ensures that the learned representation is the best suited for your task (probably you need to look into *learning theory* to know more about this). In practice, the quality of the learned feature vector will mainly depend on your available data and the inductive bias (i.e. the assumptions that you make, which are also affected by the specific neural network architecture that you choose).
  
  > **Comments:**
  > **username_0**: This may sound silly. In the answer, "In practice, the quality of the learned feature vector": is this vector the output of CNN right before we connect it to MLP or the final output.  
  > **username_1**: @NaveenKumar Yes, exactly, that's what I meant.  
  
  ---
  
  