Answers.Body: <p>For the precision metric for example you have:</p>
  
  <p><span class="math-container">$$
   Precision = \frac{TP}{TP+FP},
  $$</span>
  with TP = True Positive and FP = False Positive.</p>
  
  <p>Imagine you have the following values:<br>
  Image 1: <span class="math-container">$TP = 2, FP = 3$</span><br>
  Image 2: <span class="math-container">$TP = 1, FP = 4$</span><br>
  Image 3: <span class="math-container">$TP = 3, FP = 0$</span>  </p>
  
  <p>The precision scores as you calculated will be:<br>
  Image 1: <span class="math-container">$2/5$</span><br>
  Image 2: <span class="math-container">$1/5$</span><br>
  Image 3: <span class="math-container">$1$</span><br>
  Your average will be: <span class="math-container">$0.533$</span></p>
  
  <p>On the other hand if you sum them all up and then calculate the precision value you get:</p>
  
  <p><span class="math-container">$P = \frac{6}{6+7} = 0.462$</span></p>
  
  <p>This proves that averaging the precision scores is not the same as calculating the total precision in one go.</p>
  
  <p>Since what you want is to know how precise your algorithm is, independently of the precision for each image, you should sum all the TP and FP and only then calculate the precision for each model. <strong>This way you will not have a biased average. The average would give the same weight to an image with a larger number of objects as to another image which had fewer objects</strong>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-07T20:34:42.597Z
Answers.Text: Thank you very much for the suggestion. I think that works well than taking the mean.
Answers.UserDisplayName: Nilani Algiriyage
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-10T00:58:33.390Z
Answers.Text: @NilaniAlgiriyage You should also be careful when deciding what metric you are going for. It may depend on how your data-set is distributed (from perfectly balanced to largely imbalanced). Besides, the task at hand and target for the client may differ. F1-score, MCC (Matthews correlation coefficient for binary classification) and ROC (for comparing models) are the ones you need to consider.
Answers.UserDisplayName: bit_scientist
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-06T16:26:33.993Z
Answers.LastActivityDate: 2020-01-07T02:45:31.117Z
Answers.LastEditDate: 2020-01-07T02:45:31.117Z
Answers.UserDisplayName: Miguel Saraiva
Body: <p>I'm evaluating the accuracy in detecting objects for my image data set using three deep learning algorithms. I have selected a sample of 30 images. To measure the accuracy, I manually count the number of objects in each image and then calculate recall and precision values for three algorithms. Following is a sample:</p>
  
  <p><a href="https://i.sstatic.net/h13PQ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/h13PQ.png" alt="enter image description here"></a></p>
  
  <p>Finally to select the best model for my data set, can I calculate the mean Recall and mean Accuracy? For Example:</p>
  
  <p><a href="https://i.sstatic.net/KB10Y.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/KB10Y.png" alt="enter image description here"></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-05T22:19:56.313Z
Comments.Text: By precision do you mean IoU? Or any other else?
Comments.UserDisplayName: Clement
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-05T22:26:14.450Z
Comments.Text: @Clement Hui : Precision is simply $\frac{TP}{PP} = \frac{TP}{TP+FN}$. I have counted manually the number of objects in each image. Then I obtain the count of objects by the deep learning model.
Comments.UserDisplayName: Nilani Algiriyage
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-05T22:28:11.297Z
Comments.Text: ? What do you mean by TP PP and FN? Thanks
Comments.UserDisplayName: Clement
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-05T22:32:37.467Z
Comments.Text: @Clement Hui: It is the number of true positives(TP) over the number of predicted positives (PP). (FN) is false negatives. Thanks.
Comments.UserDisplayName: Nilani Algiriyage
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-05T22:35:35.197Z
Comments.Text: Oh ok. So isn't calculating the accuracy and precision just counting teh number of all of these and plug it in the function? Can't you do it using a program? Thanks
Comments.UserDisplayName: Clement
Tags: <deep-learning><accuracy>
Title: Can we calculate mean recall and precision
UserDisplayName: Nilani Algiriyage
ThreadText: # Can we calculate mean recall and precision
  
  (Asked by: username_0 on 2020-01-05)
  
  I'm evaluating the accuracy in detecting objects for my image data set using three deep learning algorithms. I have selected a sample of 30 images. To measure the accuracy, I manually count the number of objects in each image and then calculate recall and precision values for three algorithms. Following is a sample:
  
  [![enter image description here](https://i.sstatic.net/h13PQ.png)](https://i.sstatic.net/h13PQ.png)
  
  Finally to select the best model for my data set, can I calculate the mean Recall and mean Accuracy? For Example:
  
  [![enter image description here](https://i.sstatic.net/KB10Y.png)](https://i.sstatic.net/KB10Y.png)
  
  **Tags:** deep-learning, accuracy
  
  > **Comments:**
  > **username_1**: By precision do you mean IoU? Or any other else?  
  > **username_0**: @username_1 Hui : Precision is simply $\frac{TP}{PP} = \frac{TP}{TP+FN}$. I have counted manually the number of objects in each image. Then I obtain the count of objects by the deep learning model.  
  > **username_1**: ? What do you mean by TP PP and FN? Thanks  
  > **username_0**: @username_1 Hui: It is the number of true positives(TP) over the number of predicted positives (PP). (FN) is false negatives. Thanks.  
  > **username_1**: Oh ok. So isn't calculating the accuracy and precision just counting teh number of all of these and plug it in the function? Can't you do it using a program? Thanks  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-01-06. Score: 3 | Accepted answer
  
  For the precision metric for example you have:
  
  $$
  Precision = \frac{TP}{TP+FP},
  $$
  with TP = True Positive and FP = False Positive.
  
  Imagine you have the following values:  
  Image 1: $TP = 2, FP = 3$  
  Image 2: $TP = 1, FP = 4$  
  Image 3: $TP = 3, FP = 0$
  
  The precision scores as you calculated will be:  
  Image 1: $2/5$  
  Image 2: $1/5$  
  Image 3: $1$  
  Your average will be: $0.533$
  
  On the other hand if you sum them all up and then calculate the precision value you get:
  
  $P = \frac{6}{6+7} = 0.462$
  
  This proves that averaging the precision scores is not the same as calculating the total precision in one go.
  
  Since what you want is to know how precise your algorithm is, independently of the precision for each image, you should sum all the TP and FP and only then calculate the precision for each model. **This way you will not have a biased average. The average would give the same weight to an image with a larger number of objects as to another image which had fewer objects**.
  
  > **Comments:**
  > **username_0**: Thank you very much for the suggestion. I think that works well than taking the mean.  
  > **username_3**: @username_0 You should also be careful when deciding what metric you are going for. It may depend on how your data-set is distributed (from perfectly balanced to largely imbalanced). Besides, the task at hand and target for the client may differ. F1-score, MCC (Matthews correlation coefficient for binary classification) and ROC (for comparing models) are the ones you need to consider.  
  
  ---
  
  