Answers.Body: <p>The answer is yes this is possible and here are the papers where they do almost exactly the same project you are describing above. Although none of the bellow combine gazebo, single point/single shot, 6D-pose and CNNs. In order to use synthetic data to train a model that works on real data.</p>
  
  <ul>
  <li><a href="https://ieeexplore.ieee.org/abstract/document/8757773" rel="nofollow noreferrer">Pose Estimation by Key Points Registration in Point Cloud
  </a> (2019) By Weiyi ZHANG, Chenkun QI This paper uses gazebo to test validate the model they made.</li>
  <li><p><a href="https://arxiv.org/abs/1703.09312" rel="nofollow noreferrer">Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics</a> (2017) by Jeffrey Mahler, Jacky Liang, Sherdil Niyaz, Michael Laskey, Richard Doan, Xinyu Liu, Juan Aparicio Ojea, Ken Goldberg</p></li>
  <li><p><a href="http://openaccess.thecvf.com/content_cvpr_2018/html/Tekin_Real-Time_Seamless_Single_CVPR_2018_paper.html" rel="nofollow noreferrer">Real-Time Seamless Single Shot 6D Object Pose Prediction</a> (2018) by Bugra Tekin, Sudipta N. Sinha, Pascal Fua
  This paper sounds like exactly what you want to do.</p></li>
  <li><p><a href="http://openaccess.thecvf.com/content_iccv_2017/html/Kehl_SSD-6D_Making_RGB-Based_ICCV_2017_paper.html" rel="nofollow noreferrer">SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again</a> (2017) by Wadim Kehl, Fabian Manhardt, Federico Tombari, Slobodan Ilic, Nassir Navab 
  This paper is also referenced by them.</p></li>
  </ul>
  
  <p>The model will be able to be trained but how effect a model trained on the synthetic data will be able to properly function on real data will be the challenge. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-13T20:07:05.440Z
Answers.Text: Thanks for your reply, imo synthetic data is easier to produce, have accurate ground truth and can be manipulated by adding noise or using some smoothing to simulate real data. will definitely give it a try. Thanks again
Answers.UserDisplayName: MostafaBakr
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-13T22:52:18.893Z
Answers.Text: I definite agree with you I would just be skeptical of the effectiveness of the model in specific environments, textured shadow, detecting semi translucent objects etc, normal image detection difficulties.
Answers.UserDisplayName: Michael Hearn
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-13T18:19:08.567Z
Answers.LastActivityDate: 2019-11-13T18:19:08.567Z
Answers.UserDisplayName: Michael Hearn
Body: <p>In the case of single shot detection of point clouds, that is the point cloud of an object is taken only from one camera view without any registration. Can a Convolutional Network estimate the 6d pose of objects (initially primitive 3D objects -- cylinders, spheres, cuboids)?</p>
  <p>The dataset will be generated by simulating a depth sensor using a physics engine (ex:<a href="http://gazebosim.org" rel="nofollow noreferrer">gazebo</a>) and primitive 3D objects are spawned with known 6d pose as ground truth. The resulting training data will be the single viewed point cloud of the object with the ground truth label (6d pose)?</p>
  
Tags: <deep-learning><convolutional-neural-networks><object-recognition><object-detection><regression>
Title: Pose estimation using CNNs on Point clouds
UserDisplayName: MostafaBakr
ThreadText: # Pose estimation using CNNs on Point clouds
  
  (Asked by: username_0 on 2019-11-13)
  
  In the case of single shot detection of point clouds, that is the point cloud of an object is taken only from one camera view without any registration. Can a Convolutional Network estimate the 6d pose of objects (initially primitive 3D objects -- cylinders, spheres, cuboids)?
  
  The dataset will be generated by simulating a depth sensor using a physics engine (ex:[gazebo](http://gazebosim.org)) and primitive 3D objects are spawned with known 6d pose as ground truth. The resulting training data will be the single viewed point cloud of the object with the ground truth label (6d pose)?
  
  **Tags:** deep-learning, convolutional-neural-networks, object-recognition, object-detection, regression
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-11-13. Score: 0 | Accepted answer
  
  The answer is yes this is possible and here are the papers where they do almost exactly the same project you are describing above. Although none of the bellow combine gazebo, single point/single shot, 6D-pose and CNNs. In order to use synthetic data to train a model that works on real data.
  
  * [Pose Estimation by Key Points Registration in Point Cloud](https://ieeexplore.ieee.org/abstract/document/8757773) (2019) By Weiyi ZHANG, Chenkun QI This paper uses gazebo to test validate the model they made.
  * [Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics](https://arxiv.org/abs/1703.09312) (2017) by Jeffrey Mahler, Jacky Liang, Sherdil Niyaz, username_1 Laskey, Richard Doan, Xinyu Liu, Juan Aparicio Ojea, Ken Goldberg
  * [Real-Time Seamless Single Shot 6D Object Pose Prediction](http://openaccess.thecvf.com/content_cvpr_2018/html/Tekin_Real-Time_Seamless_Single_CVPR_2018_paper.html) (2018) by Bugra Tekin, Sudipta N. Sinha, Pascal Fua
    This paper sounds like exactly what you want to do.
  * [SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again](http://openaccess.thecvf.com/content_iccv_2017/html/Kehl_SSD-6D_Making_RGB-Based_ICCV_2017_paper.html) (2017) by Wadim Kehl, Fabian Manhardt, Federico Tombari, Slobodan Ilic, Nassir Navab
    This paper is also referenced by them.
  
  The model will be able to be trained but how effect a model trained on the synthetic data will be able to properly function on real data will be the challenge.
  
  > **Comments:**
  > **username_0**: Thanks for your reply, imo synthetic data is easier to produce, have accurate ground truth and can be manipulated by adding noise or using some smoothing to simulate real data. will definitely give it a try. Thanks again  
  > **username_1**: I definite agree with you I would just be skeptical of the effectiveness of the model in specific environments, textured shadow, detecting semi translucent objects etc, normal image detection difficulties.  
  
  ---
  
  