Answers.Body: <p>I'm not familiar with ConvNetSharp library, and the tag <code>convolutional-neural-networks</code> is a bit confusing me, but from :</p>
  
  <blockquote>
    <p>So I've created a neural net using the ConvNetSharp library which has 3 fully connected hidden layers. The first having 35 neurons and the other two having 25 neurons, each with a ReLU layer as the activation function layer.</p>
  </blockquote>
  
  <p>I assume you are building just a densely connected neural network. Correct me if I'm wrong.</p>
  
  <hr>
  
  <p>The type of neural network you need is <strong>Convolutional Neural Network</strong>.</p>
  
  <p>For image recognition (which is your case), convolutional network are almost always the answer.</p>
  
  <p>There is plenty of type of CNN, just pick one that seems appropriate and try.</p>
  
  <p>In my opinion, your task seems quite simple, you won't need really deep / complex architecture.</p>
  
  <hr>
  
  <blockquote>
    <p>It would be great if there could be a solution out there that could store a background/template using a camera and then when the camera sees that image again</p>
  </blockquote>
  
  <p>I am not aware of a model that could do what you are asking.</p>
  
  <p>But what you are asking is not really in the 'neural network' mindset. The goal of building a neural network is that you don't specify anything. The neural network will learn and find the features for you. So you just have to feed him a lot of data, and he will be able to recognize your pattern.</p>
  
  <p>Take a look at this visualization of CNN filters :</p>
  
  <p><a href="https://i.sstatic.net/SExae.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/SExae.jpg" alt="enter image description here"></a></p>
  
  <p>Here, no one gave the neural network the template of a nose or the template of an eye or the template of a face. The CNN learned it over a lot of image.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-20T16:31:12.197Z
Answers.Text: Hi Astariul, 
  I've tried using the convolutional neural network approach, but the results are not good for some reason. Maybe, its because the size of images are so small that the network isn't able to extract features and learn from them.
Answers.UserDisplayName: Hamza Abdullah
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-20T16:47:24.287Z
Answers.Text: I've updated the question to clarify things further, thanks for your consideration.
Answers.UserDisplayName: Hamza Abdullah
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-19T08:31:48.033Z
Answers.LastActivityDate: 2018-10-19T08:31:48.033Z
Answers.UserDisplayName: Astariul
Answers.Body: <p>Is it the case that one of the numbers if filled in? If so a CNN with 10 output should work well. Just choose the output that has the highest probability. If your data allows no number to be filled in, then have 11 outputs where the eleventh output indicates none is filled in. I would recommend transfer learning using the MobileNet model. Documentation is <a href="https://keras.io/applications/" rel="nofollow noreferrer">here</a>. Here is the code to adapt MobileNet to your problem:</p>
  
  <pre><code>image_size=128
  no_of_classes=10  # set to 11 if in some cases no numbers are filled in
  lr_rate=.001
  dropout=.4
  mobile = tf.keras.applications.mobilenet.MobileNet( include_top=False,
                                                             input_shape=(image_size,image_size,3),
                                                             pooling='avg', weights='imagenet',
                                                             alpha=1, depth_multiplier=1)
  x=mobile.layers[-1].output
  x=keras.layers.BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001 )(x)
  x=Dense(128, kernel_regularizer = regularizers.l2(l = 0.016),activity_regularizer=regularizers.l1(0.006),
                  bias_regularizer=regularizers.l1(0.006) ,activation='relu')(x)
  x=Dropout(rate=dropout, seed=128)(x)
  x=keras.layers.BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001 )(x)
  predictions=Dense (no_of_classes, activation='softmax')(x)
  model = Model(inputs=mobile.input, outputs=predictions)    
  for layer in model.layers:
      layer.trainable=True
  model.compile(Adam(lr=lr_rate), loss='categorical_crossentropy', metrics=['accuracy']) 
  </code></pre>
  
  <p>I would also create a training, test and validation set with about 150 images in the test set and 150 images in the validation set leaving 1700 images for training. I also recommend you use two useful callbacks. Documentation is <a href="https://keras.io/callbacks/" rel="nofollow noreferrer">here</a>. Use the ReduceLROnPlateau callback to monitor the validation loss and adjust the learning rate downward by a factor. Use ModelCheckpoint to monitor the validation loss and save the model with the lowest validation loss to use to make predictions on the test set.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-23T23:01:35.317Z
Answers.LastActivityDate: 2020-04-23T23:01:35.317Z
Answers.UserDisplayName: Gerry P
Answers.Body: <p>From what I understand, don't bother with a CNN, you have essentially perfectly structured images.</p>
  
  <p>You can hand code detectors to measure how much filled in a circle is.</p>
  
  <p>Basically do template alignment and then search over the circles.</p>
  
  <p>Ex a simple detector would measure the average blackness of the circle which you could then threshold.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-25T04:50:46.797Z
Answers.LastActivityDate: 2020-05-25T04:50:46.797Z
Answers.UserDisplayName: FourierFlux
Body: <p>I've created a neural net using the ConvNetSharp library which has 3 fully connected hidden layers. The first having 35 neurons and the other two having 25 neurons each, each layer with a ReLU layer as the activation function layer.</p>
  <p>I'm using this network for image classification - kinda. Basically it takes inputs as raw grayscale pixel values of the input image and guesses an output. I used stochastic gradient descent for the training of the model and a learning rate of 0.01. The input image is a row or column of OMR &quot;bubbles&quot; and the network has to guess which of the &quot;bubble&quot; is marked i.e filled and show the index of that bubble.</p>
  <p>I think it is because it's very hard for the network to recognize the single filled bubble among many.</p>
  <p>Here is an example image of OMR sections:</p>
  <p><a href="https://i.sstatic.net/F1cql.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/F1cql.png" alt="OMR bubbles example" /></a></p>
  <p>Using image-preprocessing, the network is given a single row or column of the above image to evaluate the marked one.</p>
  <p>Here is an example of a preprocessed image which the network sees:</p>
  <p><a href="https://i.sstatic.net/3r60U.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/3r60U.png" alt="OMR bubble column - what the network sees" /></a></p>
  <p>Here is an example of a marked input:</p>
  <p><a href="https://i.sstatic.net/LF0AO.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/LF0AO.png" alt="Marked OMR Column" /></a></p>
  <p>I've tried to use Convolutional networks but I'm not able to get them working with this.</p>
  <p><strong>What type of neural network and network architecture should I use for this kind of task? An example of such a network with code would be greatly appreciated.</strong></p>
  <p>I have tried many preprocessing techniques, such as background subtraction using the AbsDiff function in EmguCv and also using the MOG2 Algorithm, and I've also tried threshold binary function, but there still remains enough noise in the images which makes it difficult for the neural net to learn.</p>
  <p>I think this problem is not specific to using neural nets for OMR but for others too. It would be great if there could be a solution out there that could store a background/template using a camera and then when the camera sees that image again, it perspective transforms it to match exactly to the template</p>
  <p>I'm able to achieve this much - and then find their difference or do some kind of preprocessing so that a neural net could learn from it. If this is not quite possible, then is there a type of neural network out there which could detect very small features from an image and learn from it.  I have tried Convolutional Neural Network but that also isn't working very well or I'm not applying them efficiently.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-05T16:01:25.363Z
Comments.Text: What is the structure of your dataset(any bounding boxes/meta data?). Also, how many images are you using in your dataset?
Comments.UserDisplayName: Udayraj Deshmukh
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-09T12:07:44.467Z
Comments.Text: @UdayrajDeshmukh The dataset consists of a little more than 2,000 preprocessed images (as shown above) that have been labeled. No, there is no such bounding box or metadata to go with the dataset.
Comments.UserDisplayName: Hamza Abdullah
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-09T12:24:25.967Z
Comments.Text: So there are only cropped vertical strips? Also is applying ML a necessity to solve your problem?
Comments.UserDisplayName: Udayraj Deshmukh
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-10T13:38:14.760Z
Comments.Text: @UdayrajDeshmukh Sorry for the late replies.
  
  Initially, the image comes in as a full document, it is then preprocessed and cropped to these vertical strips of options/bubbles.
  
  The reason why I must use ML is the fact that the sheet is arbitrary, it can be of any color, design and the bubbles can be of any size or shape, all of this depends on the user and in the program I've provided the user with an interface to train their models. I hope that answers your question.
Comments.UserDisplayName: Hamza Abdullah
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-10T18:34:14.577Z
Comments.Text: So if OMR Checking is your problem statement. It's coincidence that I have been working on a solution for the same, and it works pretty well for document scanned images currently
Comments.UserDisplayName: Udayraj Deshmukh
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-10T18:35:28.753Z
Comments.Text: It is scheduled to be launched on product hunt soon. After that I will be able to write an answer to your question
Comments.UserDisplayName: Udayraj Deshmukh
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-11T19:06:50.337Z
Comments.Text: That's interesting. I'll be waiting for it, can you provide an estimate on the launch date maybe?.
Comments.UserDisplayName: Hamza Abdullah
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-11T19:23:14.613Z
Comments.Text: Done! Please check https://www.producthunt.com/posts/omr-checker/ 
  I can discuss the details over discord chat(link on repo). If you like it, do leave a star!
Comments.UserDisplayName: Udayraj Deshmukh
Tags: <neural-networks><deep-learning><convolutional-neural-networks><computer-vision><model-request>
Title: Which neural network to use for optical mark recognition?
UserDisplayName: Hamza Abdullah
ThreadText: # Which neural network to use for optical mark recognition?
  
  (Asked by: username_0 on 2018-10-19)
  
  I've created a neural net using the ConvNetSharp library which has 3 fully connected hidden layers. The first having 35 neurons and the other two having 25 neurons each, each layer with a ReLU layer as the activation function layer.
  
  I'm using this network for image classification - kinda. Basically it takes inputs as raw grayscale pixel values of the input image and guesses an output. I used stochastic gradient descent for the training of the model and a learning rate of 0.01. The input image is a row or column of OMR "bubbles" and the network has to guess which of the "bubble" is marked i.e filled and show the index of that bubble.
  
  I think it is because it's very hard for the network to recognize the single filled bubble among many.
  
  Here is an example image of OMR sections:
  
  [![OMR bubbles example](https://i.sstatic.net/F1cql.png)](https://i.sstatic.net/F1cql.png)
  
  Using image-preprocessing, the network is given a single row or column of the above image to evaluate the marked one.
  
  Here is an example of a preprocessed image which the network sees:
  
  [![OMR bubble column - what the network sees](https://i.sstatic.net/3r60U.png)](https://i.sstatic.net/3r60U.png)
  
  Here is an example of a marked input:
  
  [![Marked OMR Column](https://i.sstatic.net/LF0AO.png)](https://i.sstatic.net/LF0AO.png)
  
  I've tried to use Convolutional networks but I'm not able to get them working with this.
  
  **What type of neural network and network architecture should I use for this kind of task? An example of such a network with code would be greatly appreciated.**
  
  I have tried many preprocessing techniques, such as background subtraction using the AbsDiff function in EmguCv and also using the MOG2 Algorithm, and I've also tried threshold binary function, but there still remains enough noise in the images which makes it difficult for the neural net to learn.
  
  I think this problem is not specific to using neural nets for OMR but for others too. It would be great if there could be a solution out there that could store a background/template using a camera and then when the camera sees that image again, it perspective transforms it to match exactly to the template
  
  I'm able to achieve this much - and then find their difference or do some kind of preprocessing so that a neural net could learn from it. If this is not quite possible, then is there a type of neural network out there which could detect very small features from an image and learn from it. I have tried Convolutional Neural Network but that also isn't working very well or I'm not applying them efficiently.
  
  **Tags:** neural-networks, deep-learning, convolutional-neural-networks, computer-vision, model-request
  
  > **Comments:**
  > **username_1**: What is the structure of your dataset(any bounding boxes/meta data?). Also, how many images are you using in your dataset?  
  > **username_0**: @username_1 The dataset consists of a little more than 2,000 preprocessed images (as shown above) that have been labeled. No, there is no such bounding box or metadata to go with the dataset.  
  > **username_1**: So there are only cropped vertical strips? Also is applying ML a necessity to solve your problem?  
  > **username_0**: @username_1 Sorry for the late replies.  Initially, the image comes in as a full document, it is then preprocessed and cropped to these vertical strips of options/bubbles.  The reason why I must use ML is the fact that the sheet is arbitrary, it can be of any color, design and the bubbles can be of any size or shape, all of this depends on the user and in the program I've provided the user with an interface to train their models. I hope that answers your question.  
  > **username_1**: So if OMR Checking is your problem statement. It's coincidence that I have been working on a solution for the same, and it works pretty well for document scanned images currently  
  > **username_1**: It is scheduled to be launched on product hunt soon. After that I will be able to write an answer to your question  
  > **username_0**: That's interesting. I'll be waiting for it, can you provide an estimate on the launch date maybe?.  
  > **username_1**: Done! Please check https://www.producthunt.com/posts/omr-checker/  I can discuss the details over discord chat(link on repo). If you like it, do leave a star!  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_4 on 2020-05-25. Score: 2 | Accepted answer
  
  From what I understand, don't bother with a CNN, you have essentially perfectly structured images.
  
  You can hand code detectors to measure how much filled in a circle is.
  
  Basically do template alignment and then search over the circles.
  
  Ex a simple detector would measure the average blackness of the circle which you could then threshold.
  
  
  ---
  
  ### Answer by username_2 on 2018-10-19. Score: 1
  
  I'm not familiar with ConvNetSharp library, and the tag `convolutional-neural-networks` is a bit confusing me, but from :
  
  > So I've created a neural net using the ConvNetSharp library which has 3 fully connected hidden layers. The first having 35 neurons and the other two having 25 neurons, each with a ReLU layer as the activation function layer.
  
  I assume you are building just a densely connected neural network. Correct me if I'm wrong.
  
  ---
  
  The type of neural network you need is **Convolutional Neural Network**.
  
  For image recognition (which is your case), convolutional network are almost always the answer.
  
  There is plenty of type of CNN, just pick one that seems appropriate and try.
  
  In my opinion, your task seems quite simple, you won't need really deep / complex architecture.
  
  ---
  
  > It would be great if there could be a solution out there that could store a background/template using a camera and then when the camera sees that image again
  
  I am not aware of a model that could do what you are asking.
  
  But what you are asking is not really in the 'neural network' mindset. The goal of building a neural network is that you don't specify anything. The neural network will learn and find the features for you. So you just have to feed him a lot of data, and he will be able to recognize your pattern.
  
  Take a look at this visualization of CNN filters :
  
  [![enter image description here](https://i.sstatic.net/SExae.jpg)](https://i.sstatic.net/SExae.jpg)
  
  Here, no one gave the neural network the template of a nose or the template of an eye or the template of a face. The CNN learned it over a lot of image.
  
  > **Comments:**
  > **username_0**: Hi username_2,  I've tried using the convolutional neural network approach, but the results are not good for some reason. Maybe, its because the size of images are so small that the network isn't able to extract features and learn from them.  
  > **username_0**: I've updated the question to clarify things further, thanks for your consideration.  
  
  ---
  
  ### Answer by username_3 on 2020-04-23. Score: 1
  
  Is it the case that one of the numbers if filled in? If so a CNN with 10 output should work well. Just choose the output that has the highest probability. If your data allows no number to be filled in, then have 11 outputs where the eleventh output indicates none is filled in. I would recommend transfer learning using the MobileNet model. Documentation is [here](https://keras.io/applications/). Here is the code to adapt MobileNet to your problem:
  
  ```
  image_size=128
  no_of_classes=10  # set to 11 if in some cases no numbers are filled in
  lr_rate=.001
  dropout=.4
  mobile = tf.keras.applications.mobilenet.MobileNet( include_top=False,
                                                             input_shape=(image_size,image_size,3),
                                                             pooling='avg', weights='imagenet',
                                                             alpha=1, depth_multiplier=1)
  x=mobile.layers[-1].output
  x=keras.layers.BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001 )(x)
  x=Dense(128, kernel_regularizer = regularizers.l2(l = 0.016),activity_regularizer=regularizers.l1(0.006),
                  bias_regularizer=regularizers.l1(0.006) ,activation='relu')(x)
  x=Dropout(rate=dropout, seed=128)(x)
  x=keras.layers.BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001 )(x)
  predictions=Dense (no_of_classes, activation='softmax')(x)
  model = Model(inputs=mobile.input, outputs=predictions)    
  for layer in model.layers:
      layer.trainable=True
  model.compile(Adam(lr=lr_rate), loss='categorical_crossentropy', metrics=['accuracy']) 
  
  ```
  
  I would also create a training, test and validation set with about 150 images in the test set and 150 images in the validation set leaving 1700 images for training. I also recommend you use two useful callbacks. Documentation is [here](https://keras.io/callbacks/). Use the ReduceLROnPlateau callback to monitor the validation loss and adjust the learning rate downward by a factor. Use ModelCheckpoint to monitor the validation loss and save the model with the lowest validation loss to use to make predictions on the test set.
  
  
  ---
  
  