Answers.Body: <p>This is a great question. Connecting the AI knowledge to the application is a difficult task and its not easy to <strong>test and perfect</strong>.</p>
  
  <p>There are a few points regarding autonomous driving algorithms that one should keep in mind.</p>
  
  <ul>
  <li>A lot of data is required before actually applying a model to the real world.</li>
  <li>The control algorithms are designed for specific tasks, there is no <strong>master</strong> algorithm that solves the issue.</li>
  <li>Time is very important, decisions have to be made in real time.</li>
  </ul>
  
  <p>The image segmentation you have mentioned in the question is rarely used in current autonomous technologies. The main issue is that </p>
  
  <ul>
  <li>The model is slow and requires massive computing resources. Due to this is cannot help in real time decision making.</li>
  <li>A very important parameter <strong>depth</strong> is missing from the information. This is of utmost importance in autonomous driving technology. A 2D image cannot provide this parameter.</li>
  </ul>
  
  <p>So the current technology use </p>
  
  <ul>
  <li>A range of sensors (LIDAR, Ultrasonic, Cameras etc.) that provide readings 100+ times a second.</li>
  <li>A lot of the computer vision algorithms and control algorithms are hard coded. There are set parameters within which decisions are made.</li>
  <li>The control algorithms are designed to work with each set of sensors and the mechanical components of the car.</li>
  </ul>
  
  <p>To get a better look at the way these algorithms are used, you can have a look at </p>
  
  <ul>
  <li><a href="https://www.youtube.com/watch?v=FLi_IQgCxbo" rel="nofollow noreferrer">Stanley vision system</a> winner of the DARPA grand challenge (Stanford)</li>
  <li><a href="https://www.youtube.com/watch?v=tiwVMrTLUWg" rel="nofollow noreferrer">How a driverless car sees the road</a> TEDx</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-14T20:03:51.563Z
Answers.LastActivityDate: 2019-06-14T20:03:51.563Z
Answers.UserDisplayName: skillsmuggler
Body: <p>In the context of autonomous driving, two main stages are typically implemented: an image processing stage and a control stage. The first aims at extracting useful information from the acquired image while the second employs those information to control the vehicle.</p>
  
  <p>As far as concerning the processing stage, semantic segmentation is typically used. The input image is divided in different areas with a specific meaning (road, sky, car etc...). Here is an example of semantic segmentation:</p>
  
  <p><a href="https://i.sstatic.net/9skuH.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/9skuH.png" alt="enter image description here"></a></p>
  
  <p>The output of the segmentation stage is very complex. I am trying to understand how this information is typically used in the control stage, and how to use the information on the segmented areas to control the vehicle.</p>
  
  <p>For simplicity, let's just consider a vehicle that has to follow a path.</p>
  
  <p>TL;DR: <strong>what are the typical control algorithms for autonomous driving based on semantic segmentation?</strong></p>
  
Tags: <research><reference-request><autonomous-vehicles>
Title: Self-driving control logic based on semantic segmentation
UserDisplayName: firion
ThreadText: # Self-driving control logic based on semantic segmentation
  
  (Asked by: username_0 on 2018-08-23)
  
  In the context of autonomous driving, two main stages are typically implemented: an image processing stage and a control stage. The first aims at extracting useful information from the acquired image while the second employs those information to control the vehicle.
  
  As far as concerning the processing stage, semantic segmentation is typically used. The input image is divided in different areas with a specific meaning (road, sky, car etc...). Here is an example of semantic segmentation:
  
  [![enter image description here](https://i.sstatic.net/9skuH.png)](https://i.sstatic.net/9skuH.png)
  
  The output of the segmentation stage is very complex. I am trying to understand how this information is typically used in the control stage, and how to use the information on the segmented areas to control the vehicle.
  
  For simplicity, let's just consider a vehicle that has to follow a path.
  
  TL;DR: **what are the typical control algorithms for autonomous driving based on semantic segmentation?**
  
  **Tags:** research, reference-request, autonomous-vehicles
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-06-14. Score: 1
  
  This is a great question. Connecting the AI knowledge to the application is a difficult task and its not easy to **test and perfect**.
  
  There are a few points regarding autonomous driving algorithms that one should keep in mind.
  
  * A lot of data is required before actually applying a model to the real world.
  * The control algorithms are designed for specific tasks, there is no **master** algorithm that solves the issue.
  * Time is very important, decisions have to be made in real time.
  
  The image segmentation you have mentioned in the question is rarely used in current autonomous technologies. The main issue is that
  
  * The model is slow and requires massive computing resources. Due to this is cannot help in real time decision making.
  * A very important parameter **depth** is missing from the information. This is of utmost importance in autonomous driving technology. A 2D image cannot provide this parameter.
  
  So the current technology use
  
  * A range of sensors (LIDAR, Ultrasonic, Cameras etc.) that provide readings 100+ times a second.
  * A lot of the computer vision algorithms and control algorithms are hard coded. There are set parameters within which decisions are made.
  * The control algorithms are designed to work with each set of sensors and the mechanical components of the car.
  
  To get a better look at the way these algorithms are used, you can have a look at
  
  * [Stanley vision system](https://www.youtube.com/watch?v=FLi_IQgCxbo) winner of the DARPA grand challenge (Stanford)
  * [How a driverless car sees the road](https://www.youtube.com/watch?v=tiwVMrTLUWg) TEDx
  
  
  ---
  
  