Answers.Body: <p>So I am assuming that you are trying to <em>detect</em> a lego brick from the image. One idea is that you can use transfer learning. Leveraging a pre-trained machine learning model is called transfer learning. The underlying idea behind transfer learning is that one takes a well-trained model from one dataset or domain, and applies it to a new one.  François Chollet has written a very comprehensive guide to transfer learning (<a href="https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html" rel="noreferrer">https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html</a>)</p>
  
  <p>I admit I took some of this information from Christopher Bonnett's article named Classifying e-commerce products based on images and text.</p>
  
  <p>I also suggest using the Lego brick dataset from Kaggle on this link: <a href="https://www.kaggle.com/joosthazelzet/lego-brick-images" rel="noreferrer">https://www.kaggle.com/joosthazelzet/lego-brick-images</a>
  It has over 12,700 lego brick images.</p>
  
  <p>If processing power is a problem, you can use Amazon Web Services for cloud computing. It is inexpensive for small scale operations like this.</p>
  
  <p>Of course for the object detection, you can always increase the number of convolution layers. However, if you have too many layers, you should also include residual blocks/residual network. This would allow neural networks even with over a thousand layers to operate effectively. This video should help you understand how residual networks work (<a href="https://www.youtube.com/watch?v=ahkBkIGdnWQ" rel="noreferrer">https://www.youtube.com/watch?v=ahkBkIGdnWQ</a>)</p>
  
  <p>Finally, make sure not to overfit during your training and if you do follow the residual network idea, you should also include upsampling in your convolution neural network( More in here: <a href="https://towardsdatascience.com/up-sampling-with-transposed-convolution-9ae4f2df52d0" rel="noreferrer">https://towardsdatascience.com/up-sampling-with-transposed-convolution-9ae4f2df52d0</a>)</p>
  
  <p>I hope this helped and good luck on your endeavor. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-04T02:16:26.867Z
Answers.LastActivityDate: 2019-04-04T02:16:26.867Z
Answers.UserDisplayName: Hozaifa Bhutta
Body: <p>In my thesis I dealt with the question how a computer can recognize LEGO bricks. With multiple object detection, I chose a deep learning approach. I also looked at an existing training set of LEGO brick images and tried to optimize it.</p>
  
  <p><strong>My approach</strong></p>
  
  <p>By using Tensorflow's Object Detection API on a dataset of specifically generated images (Created with Blender) I was able to detect 73.3% of multiple LEGO Bricks in one Foto. </p>
  
  <p>One of the main problems I noticed was, that I tried to distinguish three different 2x4 bricks. However, colors are difficult to distinguish, especially in different lighting conditions. A better approach would have been to distinguish a 2x4 from a 2x2 and a 2x6 LEGO brick.</p>
  
  <p>Furthermore, I have noticed that the training set should best consist of "normal" and synthetically generated images. The synthetic images give variations in the lighting conditions, the backgrounds, etc., which the photographed images do not give. However, when using the trained Neural Network, photos and not synthetic images are examined. Therefore, photos should also be included in the training data set. </p>
  
  <p>One last point that would probably lead to even better results is that you train the Neural Network with pictures that show more than one LEGO brick. Because this is exactly what is required by the Neural Network when it is in use. </p>
  
  <ul>
  <li><strong>Are there other ways I could improve upon this?</strong></li>
  </ul>
  
  <p><em>(Can you see any further potential for improvement for the Neural Network? How would you approach the issue? Do any of my approaches seem poor? How do you solve the problem?)</em></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-03T12:56:56.950Z
Comments.Text: Here is another project that tries to accomplish the same. https://github.com/deeepwin/lego-cnn
Comments.UserDisplayName: Deeepwin
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-01T14:13:30.533Z
Comments.Text: Did you guys push this idea to a consumable extent? I wouldn't mind to join.
Comments.UserDisplayName: jayarjo
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-29T14:15:26.553Z
Comments.Text: Related: https://bricks.stackexchange.com/a/6124/132
Comments.UserDisplayName: Ambo100
Tags: <deep-learning><image-recognition><tensorflow><datasets><object-recognition>
Title: How to detect LEGO bricks by using a deep learning approach?
UserDisplayName: melawiki
ThreadText: # How to detect LEGO bricks by using a deep learning approach?
  
  (Asked by: username_0 on 2019-04-03)
  
  In my thesis I dealt with the question how a computer can recognize LEGO bricks. With multiple object detection, I chose a deep learning approach. I also looked at an existing training set of LEGO brick images and tried to optimize it.
  
  **My approach**
  
  By using Tensorflow's Object Detection API on a dataset of specifically generated images (Created with Blender) I was able to detect 73.3% of multiple LEGO Bricks in one Foto.
  
  One of the main problems I noticed was, that I tried to distinguish three different 2x4 bricks. However, colors are difficult to distinguish, especially in different lighting conditions. A better approach would have been to distinguish a 2x4 from a 2x2 and a 2x6 LEGO brick.
  
  Furthermore, I have noticed that the training set should best consist of "normal" and synthetically generated images. The synthetic images give variations in the lighting conditions, the backgrounds, etc., which the photographed images do not give. However, when using the trained Neural Network, photos and not synthetic images are examined. Therefore, photos should also be included in the training data set.
  
  One last point that would probably lead to even better results is that you train the Neural Network with pictures that show more than one LEGO brick. Because this is exactly what is required by the Neural Network when it is in use.
  
  * **Are there other ways I could improve upon this?**
  
  *(Can you see any further potential for improvement for the Neural Network? How would you approach the issue? Do any of my approaches seem poor? How do you solve the problem?)*
  
  **Tags:** deep-learning, image-recognition, tensorflow, datasets, object-recognition
  
  > **Comments:**
  > **username_1**: Here is another project that tries to accomplish the same. https://github.com/username_1/lego-cnn  
  > **username_2**: Did you guys push this idea to a consumable extent? I wouldn't mind to join.  
  > **username_3**: Related: https://bricks.stackexchange.com/a/6124/132  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_4 on 2019-04-04. Score: 5
  
  So I am assuming that you are trying to *detect* a lego brick from the image. One idea is that you can use transfer learning. Leveraging a pre-trained machine learning model is called transfer learning. The underlying idea behind transfer learning is that one takes a well-trained model from one dataset or domain, and applies it to a new one. François Chollet has written a very comprehensive guide to transfer learning (<https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html>)
  
  I admit I took some of this information from Christopher Bonnett's article named Classifying e-commerce products based on images and text.
  
  I also suggest using the Lego brick dataset from Kaggle on this link: <https://www.kaggle.com/joosthazelzet/lego-brick-images>
  It has over 12,700 lego brick images.
  
  If processing power is a problem, you can use Amazon Web Services for cloud computing. It is inexpensive for small scale operations like this.
  
  Of course for the object detection, you can always increase the number of convolution layers. However, if you have too many layers, you should also include residual blocks/residual network. This would allow neural networks even with over a thousand layers to operate effectively. This video should help you understand how residual networks work (<https://www.youtube.com/watch?v=ahkBkIGdnWQ>)
  
  Finally, make sure not to overfit during your training and if you do follow the residual network idea, you should also include upsampling in your convolution neural network( More in here: <https://towardsdatascience.com/up-sampling-with-transposed-convolution-9ae4f2df52d0>)
  
  I hope this helped and good luck on your endeavor.
  
  
  ---
  
  