Answers.Body: <p>Precision is the number of true positives over the number of predicted positives(PP), and recall is the number of true positives(TP) over the number of actual positives(AP) you get. I used the initials just to make it easier ahead.</p>
  
  <p>A true positive is when you predict a car in a place and there is a car in that place.</p>
  
  <p>A predicted positive is every car you predict, being right or wrong does not matter.</p>
  
  <p>A actual positive is every car that actually is in the picture.</p>
  
  <p>You should calculate these values separately for each category, and then sum over the examples you sampled, if I am not mistaken.</p>
  
  <p>So for the CAR category you have (assuming the predictions do match with the target, i.e., you are not predicting a truck as a car for example) :</p>
  
  <pre><code>model 1
  line 1  -&gt; 2 TP, 2 PP, 4 AP 
  line 2  -&gt; 0 TP, 0 PP, 2 AP
  </code></pre>
  
  <p>So in total precision  is <code>2/2 = 1</code> and recall is <code>2/6 = 0.3(3)</code>.</p>
  
  <p>You can then do the same for the other categories, and for the other models. This way you can check if a model is predicting one category better than the other. For example, model 1 can be better at finding cars in a picture whilst model 3 can be better at finding buses.</p>
  
  <p>The important part is that you know if the objects the model predicted actually correspond to what is in the picture. A very unlikely example would be a picture with 1 car and 1 truck where the algorithm recognizes the car as a truck and the truck as a car. From the info that is in the table I cannot be sure if the 2 cars you predict are the actual cars in the picture, or in other words, if they are actually True Positives or are actually False Positives. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-28T09:37:54.033Z
Answers.Text: Thanks a lot for your answer!. Yes, it is more complex than I thought. For example, there is one model which draws two bounding boxes over one car. However, your answer is so helpful!
Answers.UserDisplayName: Nilani Algiriyage
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-28T10:33:58.540Z
Answers.Text: @NilaniAlgiriyage that would make one box with higher IoU with ground truth the true positive and the other box false positive. You'd want your model to give better confidence to this TP box
Answers.UserDisplayName: deadcode
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-27T20:11:50.050Z
Answers.LastActivityDate: 2019-12-27T20:11:50.050Z
Answers.UserDisplayName: Miguel Saraiva
Body: <p>I'm using three pre-trained deep learning models to detect vehicles and count from an image data set. The vehicles belong to one of these classes ['car', 'truck', 'motorcycle', 'bus']. So, for a sample I have manually counted number of vehicles in each image. Also, I employed the three deep learning models and obtained the vehicle counts. For example:</p>
  
  <pre><code>    Actual        | model 1 count| model 2 count  | model 3 count 
  ------------------------------------------------------------------
      4 cars, 1 bus | 2 cars       | 2 cars, 1 truck| 4 cars
      2 cars        | 0            | 1 truck        | 1 car, 1 bus
  </code></pre>
  
  <p>In this case, how can I measure accuracy scores such as precision and recall?</p>
  
Tags: <deep-learning><convolutional-neural-networks><object-detection><precision><recall>
Title: How to calculate the precision and recall given the predictions and targets in this case?
UserDisplayName: Nilani Algiriyage
ThreadText: # How to calculate the precision and recall given the predictions and targets in this case?
  
  (Asked by: username_0 on 2019-12-27)
  
  I'm using three pre-trained deep learning models to detect vehicles and count from an image data set. The vehicles belong to one of these classes ['car', 'truck', 'motorcycle', 'bus']. So, for a sample I have manually counted number of vehicles in each image. Also, I employed the three deep learning models and obtained the vehicle counts. For example:
  
  ```
      Actual        | model 1 count| model 2 count  | model 3 count 
  ------------------------------------------------------------------
      4 cars, 1 bus | 2 cars       | 2 cars, 1 truck| 4 cars
      2 cars        | 0            | 1 truck        | 1 car, 1 bus
  
  ```
  
  In this case, how can I measure accuracy scores such as precision and recall?
  
  **Tags:** deep-learning, convolutional-neural-networks, object-detection, precision, recall
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-12-27. Score: 2 | Accepted answer
  
  Precision is the number of true positives over the number of predicted positives(PP), and recall is the number of true positives(TP) over the number of actual positives(AP) you get. I used the initials just to make it easier ahead.
  
  A true positive is when you predict a car in a place and there is a car in that place.
  
  A predicted positive is every car you predict, being right or wrong does not matter.
  
  A actual positive is every car that actually is in the picture.
  
  You should calculate these values separately for each category, and then sum over the examples you sampled, if I am not mistaken.
  
  So for the CAR category you have (assuming the predictions do match with the target, i.e., you are not predicting a truck as a car for example) :
  
  ```
  model 1
  line 1  -> 2 TP, 2 PP, 4 AP 
  line 2  -> 0 TP, 0 PP, 2 AP
  
  ```
  
  So in total precision is `2/2 = 1` and recall is `2/6 = 0.3(3)`.
  
  You can then do the same for the other categories, and for the other models. This way you can check if a model is predicting one category better than the other. For example, model 1 can be better at finding cars in a picture whilst model 3 can be better at finding buses.
  
  The important part is that you know if the objects the model predicted actually correspond to what is in the picture. A very unlikely example would be a picture with 1 car and 1 truck where the algorithm recognizes the car as a truck and the truck as a car. From the info that is in the table I cannot be sure if the 2 cars you predict are the actual cars in the picture, or in other words, if they are actually True Positives or are actually False Positives.
  
  > **Comments:**
  > **username_0**: Thanks a lot for your answer!. Yes, it is more complex than I thought. For example, there is one model which draws two bounding boxes over one car. However, your answer is so helpful!  
  > **username_2**: @username_0 that would make one box with higher IoU with ground truth the true positive and the other box false positive. You'd want your model to give better confidence to this TP box  
  
  ---
  
  