Answers.Body: <p>It seems that LIDAR presents a problem for resolving the car's environment at higher speed. While I'm not too familiar with the dynamics of LIDAR I do know that <a href="https://www.edn.com/design/analog/4442319/Autonomous-automotive-sensors--How-processor-algorithms-get-their-inputs" rel="nofollow noreferrer">it's a physical system that relies on sending and receiving laser pulses to various points around the car by way of rotating mirrors</a>. As speeds increase, it seems <a href="https://en.wikipedia.org/wiki/Lidar#Autonomous%20vehicles" rel="nofollow noreferrer">different arrangements of mirrors and light collectors might have to be used</a> to maintain a high-resolution image. There's some evidence that <a href="ftp://ftp.rap.ucar.edu/pub/rgf/velerr.pdf" rel="nofollow noreferrer">Doppler LIDAR (developed in the 1990s) became less accurate with higher velocities.</a> However, LIDAR is partly preferred over radar because of its higher accuracy even when tracking objects at high speeds - <a href="https://www.officer.com/on-the-street/article/10250592/lidar-the-speed-enforcement-weapon-of-choice" rel="nofollow noreferrer">this is why LIDAR guns are increasingly being used by police instead of radar guns to track speeding vehicles</a>. It seems natural that a set of high-resolution cameras paired with a well-trained neural network would not be subject to the same physical limitations as LIDAR.</p>
  
  <p>I think that an important intuition to consider is that while LIDAR is used to generate clouds of data points whose shapes and patterns can be analyzed by autonomous car software, cameras can pick up non-topograhical features such as road lines, the content of roadsigns, and additional location context such as storefronts and intersection layouts. Considering that these cameras can use pattern recognition and stereoscopy to also generate a 3D topographic map of the environment, it seems plausible that Level 5 self driving cars would not require LIDAR.</p>
  
  <p><a href="https://seekingalpha.com/article/4106093-tesla-betting-cameras-full-self-driving" rel="nofollow noreferrer">Here's an interesting look at the problem.</a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-12-25T07:19:49.200Z
Answers.LastActivityDate: 2017-12-25T07:19:49.200Z
Answers.UserDisplayName: optimizedp
Answers.Body: <p>Given that a self-driving car is trying to replicate the performance of a two-camera system (or one in a pinch), there is nothing in principal that mandates lidar for a self-driving car.  Lidar is a shortcut, substituting sensor sophistication for image-processing sophistication.  AFAIK <a href="https://www.youtube.com/watch?v=-96BEoXJMs0" rel="nofollow noreferrer">Nvidia's own self-driving vehicle</a>  doesn't have Lidar.  My personal opinion is that Level 5 self-driving vehicles won't be practical until they have the kind of image-processing sophistication that makes Lidar an unnecessary crutch. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-10-24T15:15:40.080Z
Answers.LastActivityDate: 2017-10-24T15:15:40.080Z
Answers.UserDisplayName: antlersoft
Body: <p>I have a liberal arts background so I need help understanding <a href="https://arxiv.org/pdf/1708.09839.pdf" rel="nofollow noreferrer">this paper</a>, particularly pages 26 to 30. The authors test a four-camera system for localization, mapping, and obstacle detection for self-driving cars. The paper seems to say the multi-camera system can map the environment to within an average of 7 cm (2.8 inches) of accuracy (with the largest error being 16 cm or 6.3) and detect obstacles to within 10 cm (3.9 inches) of accuracy. Am I getting this right?</p>
  
  <p>Given that automotive lidar can detect objects to within 1.5 cm (0.6 inches) of accuracy, and given that for driving purposes the difference between 1.5 cm and 7 cm, 10 cm, or 16 cm seems quite small, can a multi-camera system be used instead of lidar in a self-driving car application? How do driving speeds affect things? What crucial elements of the problem space might I be overlooking or misunderstanding?   </p>
  
Tags: <autonomous-vehicles>
Title: ‪Can a multi-camera system be used for localization, mapping, and obstacle detection in self-driving cars to within 10 cm of accuracy? Whither lidar?‬
UserDisplayName: T.A. Eady
ThreadText: # ‪Can a multi-camera system be used for localization, mapping, and obstacle detection in self-driving cars to within 10 cm of accuracy? Whither lidar?‬
  
  (Asked by: username_0 on 2017-10-24)
  
  I have a liberal arts background so I need help understanding [this paper](https://arxiv.org/pdf/1708.09839.pdf), particularly pages 26 to 30. The authors test a four-camera system for localization, mapping, and obstacle detection for self-driving cars. The paper seems to say the multi-camera system can map the environment to within an average of 7 cm (2.8 inches) of accuracy (with the largest error being 16 cm or 6.3) and detect obstacles to within 10 cm (3.9 inches) of accuracy. Am I getting this right?
  
  Given that automotive lidar can detect objects to within 1.5 cm (0.6 inches) of accuracy, and given that for driving purposes the difference between 1.5 cm and 7 cm, 10 cm, or 16 cm seems quite small, can a multi-camera system be used instead of lidar in a self-driving car application? How do driving speeds affect things? What crucial elements of the problem space might I be overlooking or misunderstanding?
  
  **Tags:** autonomous-vehicles
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2017-12-25. Score: 1
  
  It seems that LIDAR presents a problem for resolving the car's environment at higher speed. While I'm not too familiar with the dynamics of LIDAR I do know that [it's a physical system that relies on sending and receiving laser pulses to various points around the car by way of rotating mirrors](https://www.edn.com/design/analog/4442319/Autonomous-automotive-sensors--How-processor-algorithms-get-their-inputs). As speeds increase, it seems [different arrangements of mirrors and light collectors might have to be used](https://en.wikipedia.org/wiki/Lidar#Autonomous%20vehicles) to maintain a high-resolution image. There's some evidence that [Doppler LIDAR (developed in the 1990s) became less accurate with higher velocities.](ftp://ftp.rap.ucar.edu/pub/rgf/velerr.pdf) However, LIDAR is partly preferred over radar because of its higher accuracy even when tracking objects at high speeds - [this is why LIDAR guns are increasingly being used by police instead of radar guns to track speeding vehicles](https://www.officer.com/on-the-street/article/10250592/lidar-the-speed-enforcement-weapon-of-choice). It seems natural that a set of high-resolution cameras paired with a well-trained neural network would not be subject to the same physical limitations as LIDAR.
  
  I think that an important intuition to consider is that while LIDAR is used to generate clouds of data points whose shapes and patterns can be analyzed by autonomous car software, cameras can pick up non-topograhical features such as road lines, the content of roadsigns, and additional location context such as storefronts and intersection layouts. Considering that these cameras can use pattern recognition and stereoscopy to also generate a 3D topographic map of the environment, it seems plausible that Level 5 self driving cars would not require LIDAR.
  
  [Here's an interesting look at the problem.](https://seekingalpha.com/article/4106093-tesla-betting-cameras-full-self-driving)
  
  
  ---
  
  ### Answer by username_2 on 2017-10-24. Score: 0
  
  Given that a self-driving car is trying to replicate the performance of a two-camera system (or one in a pinch), there is nothing in principal that mandates lidar for a self-driving car. Lidar is a shortcut, substituting sensor sophistication for image-processing sophistication. AFAIK [Nvidia's own self-driving vehicle](https://www.youtube.com/watch?v=-96BEoXJMs0) doesn't have Lidar. My personal opinion is that Level 5 self-driving vehicles won't be practical until they have the kind of image-processing sophistication that makes Lidar an unnecessary crutch.
  
  
  ---
  
  