Answers.Body: <p>I highly recommend you read the seminal work on convolutional neural networks (CNNs):</p>
  <p>Lecun, Y., &amp; Bengio, Y. (1995). Convolutional networks for images, speech, and time-series. In M. A. Arbib (Ed.), The handbook of brain theory and neural networks MIT Press.</p>
  <p>In summary, CNNs provide the following advantages over feedforward neural networks:</p>
  <ul>
  <li><p><strong>Local Connectivity</strong>: In CNNs, each neuron in a layer is connected
  only to a small region of the input. This reflects the way images
  are structured, where local groups of pixels are often strongly
  correlated, indicating the presence of specific local features like
  edges, textures, or colors. This local connectivity reduces the
  number of parameters in the network, making it more computationally
  efficient.</p>
  </li>
  <li><p><strong>Shared Weights and Translation Invariance</strong>: CNNs use the same weights
  across different parts of the input, a concept known as weight
  sharing. This approach not only reduces the number of parameters but
  also provides the network with a form of translation invariance,
  meaning the network can recognize an object regardless of its
  location in the image.</p>
  </li>
  <li><p><strong>Hierarchical Feature Learning</strong>: CNNs are designed to automatically
  and adaptively learn spatial hierarchies of features from input
  images. The layers in a CNN typically detect edges in the initial
  layers, shapes in the middle layers, and high-level content like
  object parts in the deeper layers. This hierarchical structure is
  particularly suited for image data, as images often contain
  hierarchical patterns.</p>
  </li>
  <li><p><strong>Reduction in Overfitting</strong>: Due to their architecture, CNNs tend to
  have fewer parameters than fully connected feedforward networks of
  similar depth, which reduces the risk of overfitting on image
  datasets. This is especially important in image recognition tasks,
  where the input data can be very high-dimensional (like
  high-resolution images).</p>
  </li>
  <li><p><strong>Efficiency in Image Processing</strong>: CNNs are particularly efficient for
  image processing tasks. The combination of local connectivity and
  shared weights makes them more suited to handle the high
  dimensionality of raw image data compared to fully connected
  networks, which can become impractically large for image inputs.</p>
  </li>
  </ul>
  <p><a href="https://i.sstatic.net/KxocZ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/KxocZ.png" alt="enter image description here" /></a>
  <strong>Typical Layers in a CNN</strong></p>
  <ol>
  <li><strong>Input Layer</strong>: This is where the network receives the image data. Images are typically represented as a 3D array of size height x width x depth (depth refers to color channels, e.g., RGB).</li>
  <li><strong>Convolutional Layer</strong>: The core building block of a CNN that performs most of the computational heavy lifting. This layer uses filters (or kernels) to convolve the input and produce feature maps. Each filter detects features such as edges, corners, etc.</li>
  <li><strong>Activation Layer (ReLU)</strong>: After each convolution operation, the activation layer, typically a Rectified Linear Unit (ReLU), is applied to introduce non-linearity into the model, allowing it to learn more complex patterns.</li>
  <li><strong>Pooling Layer</strong>: This layer is used to reduce the spatial size (width and height) of the input volume for the next convolutional layer. It helps decrease the computational power required to process the data through dimensionality reduction. Max pooling is the most common type.</li>
  <li><strong>Fully Connected Layer (FC)</strong>: Neurons in a fully connected layer have full connections to all activations in the previous layer, as seen in regular Neural Networks. These layers are typically placed near the end of CNN architectures and can be used to optimize objectives like class scores.</li>
  <li><strong>Output Layer</strong>: The final layer, often a fully connected layer, outputs the final predictions of the network. For classification tasks, this might be a softmax layer that provides probabilities for each class.</li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-17T17:36:34.413Z
Answers.LastActivityDate: 2023-12-17T17:36:34.413Z
Answers.UserDisplayName: Brian O'Donnell
Answers.Body: <p>(This is <a href="https://stats.stackexchange.com/a/409172/247274">taken verbatim</a> from a post of mine on the statistics Stack, Cross Validated.)</p>
  <p>I think CNNs are often talked about as putting squares on top of bigger squares with the &quot;neural network&quot; aspect hidden. They're definitely neural networks and can be drawn out.</p>
  <p>Apply the filter to the upper left 2x2 array.</p>
  <p><a href="https://i.sstatic.net/jWSzh.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/jWSzh.png" alt="Apply the filter to the upper left 2x2 array." /></a></p>
  <p>Apply the filter to the upper right 2x2 array.</p>
  <p><a href="https://i.sstatic.net/j52Ca.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/j52Ca.png" alt="enter image description here" /></a></p>
  <p>Apply the filter to the bottom left 2x2 array.</p>
  <p><a href="https://i.sstatic.net/UFB8t.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/UFB8t.png" alt="enter image description here" /></a></p>
  <p>Apply the filter to the bottom right 2x2 array.</p>
  <p><a href="https://i.sstatic.net/y6vdB.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/y6vdB.png" alt="enter image description here" /></a></p>
  <p>Here is the entire layer, with the 3x3 input image mapping to four neurons for the four positions in the image where convolution occurs.</p>
  <p><a href="https://i.sstatic.net/lo0sm.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/lo0sm.png" alt="enter image description here" /></a></p>
  <p>You can draw those four neurons in a square array if you plan to do additional convolutional layers. That doesn't make so much sense with a 2x2 output, but you're probably working with images that are bigger than 3x3.</p>
  <p>I think that it's a useful exercise to draw out a simple example like this. Another useful exercise is to predict how many parameters there will be in a layer of 3x3 filtering (10 filters) over a 5x5 image. (The answer is 100: 9 for each of the ten filters, plus one bias term per filter, but bias terms typically get ignored when we draw out the &quot;web&quot; of a neural network. (However, bias terms are important!) Now try it for 3x4 filtering (20 filters) over a 7x5 image!)</p>
  <p>To compare with a fully-connected layer, the fully-connected layer would have each pixel linked to each of the four neurons in the next layer, and each of those <span class="math-container">$36$</span> connections would have its own color to indicate the connection having its own weight, rather than sharing weights as is done in a convolutional layer.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-17T18:27:15.663Z
Answers.LastActivityDate: 2023-12-22T16:46:57.487Z
Answers.LastEditDate: 2023-12-22T16:46:57.487Z
Answers.UserDisplayName: Dave
Body: <p>Why do we use Convolutional Neural Network (CNN) for image data and not the Feedforward Neural Network (FNN)? Draw and explain the architecture of Convolutional Network</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-12-17T17:36:50.483Z
Comments.Text: I’m voting to close this question because it clearly looks like a homework question where you have not show any research effort to solve it and just wants us to give you a solution.
Comments.UserDisplayName: nbro
Tags: <deep-learning>
Title: Why we use Convolutional Neural Network for image data and not the Feedforward Neural Network Draw and explain the architecture of Convolutional Netwo
UserDisplayName: Himanshu Sinha
ThreadText: # Why we use Convolutional Neural Network for image data and not the Feedforward Neural Network Draw and explain the architecture of Convolutional Netwo
  
  (Asked by: username_0 on 2023-12-15)
  
  Why do we use Convolutional Neural Network (CNN) for image data and not the Feedforward Neural Network (FNN)? Draw and explain the architecture of Convolutional Network
  
  **Tags:** deep-learning
  
  > **Comments:**
  > **username_1**: I’m voting to close this question because it clearly looks like a homework question where you have not show any research effort to solve it and just wants us to give you a solution.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2023-12-17. Score: 1
  
  I highly recommend you read the seminal work on convolutional neural networks (CNNs):
  
  Lecun, Y., & Bengio, Y. (1995). Convolutional networks for images, speech, and time-series. In M. A. Arbib (Ed.), The handbook of brain theory and neural networks MIT Press.
  
  In summary, CNNs provide the following advantages over feedforward neural networks:
  
  * **Local Connectivity**: In CNNs, each neuron in a layer is connected
    only to a small region of the input. This reflects the way images
    are structured, where local groups of pixels are often strongly
    correlated, indicating the presence of specific local features like
    edges, textures, or colors. This local connectivity reduces the
    number of parameters in the network, making it more computationally
    efficient.
  * **Shared Weights and Translation Invariance**: CNNs use the same weights
    across different parts of the input, a concept known as weight
    sharing. This approach not only reduces the number of parameters but
    also provides the network with a form of translation invariance,
    meaning the network can recognize an object regardless of its
    location in the image.
  * **Hierarchical Feature Learning**: CNNs are designed to automatically
    and adaptively learn spatial hierarchies of features from input
    images. The layers in a CNN typically detect edges in the initial
    layers, shapes in the middle layers, and high-level content like
    object parts in the deeper layers. This hierarchical structure is
    particularly suited for image data, as images often contain
    hierarchical patterns.
  * **Reduction in Overfitting**: Due to their architecture, CNNs tend to
    have fewer parameters than fully connected feedforward networks of
    similar depth, which reduces the risk of overfitting on image
    datasets. This is especially important in image recognition tasks,
    where the input data can be very high-dimensional (like
    high-resolution images).
  * **Efficiency in Image Processing**: CNNs are particularly efficient for
    image processing tasks. The combination of local connectivity and
    shared weights makes them more suited to handle the high
    dimensionality of raw image data compared to fully connected
    networks, which can become impractically large for image inputs.
  
  [![enter image description here](https://i.sstatic.net/KxocZ.png)](https://i.sstatic.net/KxocZ.png)
  **Typical Layers in a CNN**
  
  1. **Input Layer**: This is where the network receives the image data. Images are typically represented as a 3D array of size height x width x depth (depth refers to color channels, e.g., RGB).
  2. **Convolutional Layer**: The core building block of a CNN that performs most of the computational heavy lifting. This layer uses filters (or kernels) to convolve the input and produce feature maps. Each filter detects features such as edges, corners, etc.
  3. **Activation Layer (ReLU)**: After each convolution operation, the activation layer, typically a Rectified Linear Unit (ReLU), is applied to introduce non-linearity into the model, allowing it to learn more complex patterns.
  4. **Pooling Layer**: This layer is used to reduce the spatial size (width and height) of the input volume for the next convolutional layer. It helps decrease the computational power required to process the data through dimensionality reduction. Max pooling is the most common type.
  5. **Fully Connected Layer (FC)**: Neurons in a fully connected layer have full connections to all activations in the previous layer, as seen in regular Neural Networks. These layers are typically placed near the end of CNN architectures and can be used to optimize objectives like class scores.
  6. **Output Layer**: The final layer, often a fully connected layer, outputs the final predictions of the network. For classification tasks, this might be a softmax layer that provides probabilities for each class.
  
  
  ---
  
  ### Answer by username_3 on 2023-12-17. Score: 1
  
  (This is [taken verbatim](https://stats.stackexchange.com/a/409172/247274) from a post of mine on the statistics Stack, Cross Validated.)
  
  I think CNNs are often talked about as putting squares on top of bigger squares with the "neural network" aspect hidden. They're definitely neural networks and can be drawn out.
  
  Apply the filter to the upper left 2x2 array.
  
  [![Apply the filter to the upper left 2x2 array.](https://i.sstatic.net/jWSzh.png)](https://i.sstatic.net/jWSzh.png)
  
  Apply the filter to the upper right 2x2 array.
  
  [![enter image description here](https://i.sstatic.net/j52Ca.png)](https://i.sstatic.net/j52Ca.png)
  
  Apply the filter to the bottom left 2x2 array.
  
  [![enter image description here](https://i.sstatic.net/UFB8t.png)](https://i.sstatic.net/UFB8t.png)
  
  Apply the filter to the bottom right 2x2 array.
  
  [![enter image description here](https://i.sstatic.net/y6vdB.png)](https://i.sstatic.net/y6vdB.png)
  
  Here is the entire layer, with the 3x3 input image mapping to four neurons for the four positions in the image where convolution occurs.
  
  [![enter image description here](https://i.sstatic.net/lo0sm.png)](https://i.sstatic.net/lo0sm.png)
  
  You can draw those four neurons in a square array if you plan to do additional convolutional layers. That doesn't make so much sense with a 2x2 output, but you're probably working with images that are bigger than 3x3.
  
  I think that it's a useful exercise to draw out a simple example like this. Another useful exercise is to predict how many parameters there will be in a layer of 3x3 filtering (10 filters) over a 5x5 image. (The answer is 100: 9 for each of the ten filters, plus one bias term per filter, but bias terms typically get ignored when we draw out the "web" of a neural network. (However, bias terms are important!) Now try it for 3x4 filtering (20 filters) over a 7x5 image!)
  
  To compare with a fully-connected layer, the fully-connected layer would have each pixel linked to each of the four neurons in the next layer, and each of those $36$ connections would have its own color to indicate the connection having its own weight, rather than sharing weights as is done in a convolutional layer.
  
  
  ---
  
  