Answers.Body: <p>If you are using a fully connected network (aka an <a href="https://en.wikipedia.org/wiki/Multilayer_perceptron" rel="nofollow noreferrer">MLP</a>) and images with one channel (grey scale) and 100 x 100 = 10,000 pixels, then yes, MLP would have 10k inputs and 10k x N <sup>1</sup> trainable weights in the first layer (as noted by <a href="https://ai.stackexchange.com/users/1847/neil-slater">Neil Slater</a>). If you have a color image with 3 channels, e.g. RGB, then you can expect 3 times as many weights because there are 3 times as many values used to represent the image.</p>
  
  <p>A <a href="http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networ" rel="nofollow noreferrer">convolutional neural network</a> is a common architecture for analyzing images. For a 100x100 (10k) pixel image, the input layer might have 3x3x1x32 = 288 weights (for 1 channel) or 3x3x3x32 = 864 weights in the first layer, much less than 10k x N <sup>1</sup> from a fully connected network. This would transform your image into a 98x98x32 size image. The main point is that you would have 3x3 weights per input channel per output channel at each layer, instead of 10k weights per input channel per output channel. CNNs also give you some <a href="https://en.wikipedia.org/wiki/Convolutional_neural_network" rel="nofollow noreferrer">invariance properties</a> that are usually nice in machine learning with images.</p>
  
  <p>For images in general, having a lot of weights is normal. ImageNet (linked above) has 60 million weights to train. Typically special hardware, like a <a href="https://en.wikipedia.org/wiki/Graphics_processing_unit" rel="nofollow noreferrer">GPU</a> is used to handle this many weights. If you are using just a CPU, your model may not train well in any reasonable amount of time, i.e. years.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-10T09:00:39.957Z
Answers.Text: A fully-connected network would actually have $10000 \times N^1$ weights where $N^1$ is the number of neurons in the first hidden layer. Assuming your CNN example is expanding number of channels from 3 to 32, its 288 weights when fully converted into a $3 \times 10000$ input feeding a $32 \times 10000$ first hidden layer, would have 9.6 billion weights to train!
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-14T19:23:00.600Z
Answers.Text: edited to incorporate the $10000 \times N^1$. @NeilSlater how do you get 9.6 billion?
Answers.UserDisplayName: bjschoenfeld
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-14T19:55:39.723Z
Answers.Text: 3 channels times 10,000 pixels input times 32 channels times 10,000 "feature pixels" in first layer.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-14T22:14:42.363Z
Answers.Text: I see. You are saying that while a CNN layer would have 288 (or really 864) weights, while a fully connected layer would have 9.6 billion. A good point!
Answers.UserDisplayName: bjschoenfeld
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-10T02:27:01.950Z
Answers.LastActivityDate: 2019-08-14T19:18:35.740Z
Answers.LastEditDate: 2019-08-14T19:18:35.740Z
Answers.UserDisplayName: bjschoenfeld
Body: <p>I am trying to understand the best practice to read and analyze images. If your image has 10,000 pixels, your input layers will have 10,000 inputs?</p>
  
  <p>It sounds that my neural network will have too many inputs if I do it that way. Is that a problem? What is the recommended way of feeding an image through a neural network?</p>
  
Tags: <neural-networks><deep-learning><convolutional-neural-networks><computer-vision>
Title: What is the correct way to read and analyse images in machine learning?
UserDisplayName: Eddie Bravo
ThreadText: # What is the correct way to read and analyse images in machine learning?
  
  (Asked by: username_0 on 2019-08-09)
  
  I am trying to understand the best practice to read and analyze images. If your image has 10,000 pixels, your input layers will have 10,000 inputs?
  
  It sounds that my neural network will have too many inputs if I do it that way. Is that a problem? What is the recommended way of feeding an image through a neural network?
  
  **Tags:** neural-networks, deep-learning, convolutional-neural-networks, computer-vision
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-08-10. Score: 1
  
  If you are using a fully connected network (aka an [MLP](https://en.wikipedia.org/wiki/Multilayer_perceptron)) and images with one channel (grey scale) and 100 x 100 = 10,000 pixels, then yes, MLP would have 10k inputs and 10k x N 1 trainable weights in the first layer (as noted by [Neil Slater](https://ai.stackexchange.com/users/1847/neil-slater)). If you have a color image with 3 channels, e.g. RGB, then you can expect 3 times as many weights because there are 3 times as many values used to represent the image.
  
  A [convolutional neural network](http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networ) is a common architecture for analyzing images. For a 100x100 (10k) pixel image, the input layer might have 3x3x1x32 = 288 weights (for 1 channel) or 3x3x3x32 = 864 weights in the first layer, much less than 10k x N 1 from a fully connected network. This would transform your image into a 98x98x32 size image. The main point is that you would have 3x3 weights per input channel per output channel at each layer, instead of 10k weights per input channel per output channel. CNNs also give you some [invariance properties](https://en.wikipedia.org/wiki/Convolutional_neural_network) that are usually nice in machine learning with images.
  
  For images in general, having a lot of weights is normal. ImageNet (linked above) has 60 million weights to train. Typically special hardware, like a [GPU](https://en.wikipedia.org/wiki/Graphics_processing_unit) is used to handle this many weights. If you are using just a CPU, your model may not train well in any reasonable amount of time, i.e. years.
  
  > **Comments:**
  > **username_2**: A fully-connected network would actually have $10000 \times N^1$ weights where $N^1$ is the number of neurons in the first hidden layer. Assuming your CNN example is expanding number of channels from 3 to 32, its 288 weights when fully converted into a $3 \times 10000$ input feeding a $32 \times 10000$ first hidden layer, would have 9.6 billion weights to train!  
  > **username_1**: edited to incorporate the $10000 \times N^1$. @NeilSlater how do you get 9.6 billion?  
  > **username_2**: 3 channels times 10,000 pixels input times 32 channels times 10,000 "feature pixels" in first layer.  
  > **username_1**: I see. You are saying that while a CNN layer would have 288 (or really 864) weights, while a fully connected layer would have 9.6 billion. A good point!  
  
  ---
  
  