Answers.Body: <p><strong>Note</strong> that in the <a href="https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-1037-6" rel="noreferrer">article published</a> that figure is different from the preprint!<br>
  See the figure:
  <a href="https://i.sstatic.net/QmrVD.gif" rel="noreferrer"><img src="https://i.sstatic.net/QmrVD.gif" alt="Figure 3 of the paper equivalent to the figure 8 of the preprint"></a></p>
  
  <p>As they aren't substantially different in display other than showing the two set of predictions that are comparable between CAFA1 and CAFA2 I explain them below:</p>
  
  <p>The colors are the margins of $F_{max}$ they refer to the equation found in the section "Comparison between CAFA1 and CAFA2 methods" under methods:</p>
  
  <p>$$\delta(m_2, m_1) = \frac{1}{B}\sum_{b=1}^{B}F_{max}^{b}(m_2)-\frac{1}{B}\sum_{b=1}^{B}F_{max}^{b}(m_1)$$</p>
  
  <p>Where B is the 10000 bootstraps they did and $m_1$ and $m_2$ are the CAFA1 and CAFA2 methods. </p>
  
  <p>Where $F_{max}$ is explained in the article (and in the preprint) as:</p>
  
  <blockquote>
    <p>A perfect predictor would be characterized with $F_{max} = 1 $</p>
  </blockquote>
  
  <p>So, the higher the (positive) difference is, the better precision and recall (<a href="https://en.wikipedia.org/wiki/F1_score" rel="noreferrer">See wikipedia</a>) are for those methods. Note that the scale is from -0.1 to 0.1 not from -1 to 1 so the differences are quite small (One could argue that the log10 would be better to see those small differences)</p>
  
  <p>One can appreciate that in the bar plots at the right, where the differences between each dataset are small in the B section, so the color is white, and in the C panels they have brighter color. </p>
  
  <p>The numbers in the boxes are the percentage that a CAFA2 method did better than a method in CAFA1 for another dataset in the "intersection of the two GO snapshots" as noted in the methods section. (But it is also not clear to me)</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-12-04T22:10:15.407Z
Answers.LastActivityDate: 2017-12-04T22:12:25.767Z
Answers.LastEditDate: 2017-12-04T22:12:25.767Z
Answers.UserDisplayName: llrs
Body: <p>I have a question about CAFA1 and CAFA2 experiment. </p>
  
  <p>CAFA is an experiment that compare computational methods predicting protein functions. In <a href="https://arxiv.org/abs/1601.00891" rel="nofollow noreferrer">the paper</a> discussing CAFA2, there is an image comparing 2 methods (CAFA1 versus CAFA2):</p>
  
  <p><a href="https://i.sstatic.net/xMbXq.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/xMbXq.png" alt="Figure 8 from paper"></a></p>
  
  <p>How do I interpret the numbers and colors (dark and light)? What do they mean?</p>
  
Tags: <figure-explanation>
Title: About CAFA experiment
ThreadText: # About CAFA experiment
  
  (Asked by: username_0 on 2017-12-04)
  
  I have a question about CAFA1 and CAFA2 experiment.
  
  CAFA is an experiment that compare computational methods predicting protein functions. In [the paper](https://arxiv.org/abs/1601.00891) discussing CAFA2, there is an image comparing 2 methods (CAFA1 versus CAFA2):
  
  [![Figure 8 from paper](https://i.sstatic.net/xMbXq.png)](https://i.sstatic.net/xMbXq.png)
  
  How do I interpret the numbers and colors (dark and light)? What do they mean?
  
  **Tags:** figure-explanation
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2017-12-04. Score: 5
  
  **Note** that in the [article published](https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-1037-6) that figure is different from the preprint!  
  See the figure:
  [![Figure 3 of the paper equivalent to the figure 8 of the preprint](https://i.sstatic.net/QmrVD.gif)](https://i.sstatic.net/QmrVD.gif)
  
  As they aren't substantially different in display other than showing the two set of predictions that are comparable between CAFA1 and CAFA2 I explain them below:
  
  The colors are the margins of $F\_{max}$ they refer to the equation found in the section "Comparison between CAFA1 and CAFA2 methods" under methods:
  
  $$\delta(m\_2, m\_1) = \frac{1}{B}\sum\_{b=1}^{B}F\_{max}^{b}(m\_2)-\frac{1}{B}\sum\_{b=1}^{B}F\_{max}^{b}(m\_1)$$
  
  Where B is the 10000 bootstraps they did and $m\_1$ and $m\_2$ are the CAFA1 and CAFA2 methods.
  
  Where $F\_{max}$ is explained in the article (and in the preprint) as:
  
  > A perfect predictor would be characterized with $F\_{max} = 1 $
  
  So, the higher the (positive) difference is, the better precision and recall ([See wikipedia](https://en.wikipedia.org/wiki/F1_score)) are for those methods. Note that the scale is from -0.1 to 0.1 not from -1 to 1 so the differences are quite small (One could argue that the log10 would be better to see those small differences)
  
  One can appreciate that in the bar plots at the right, where the differences between each dataset are small in the B section, so the color is white, and in the C panels they have brighter color.
  
  The numbers in the boxes are the percentage that a CAFA2 method did better than a method in CAFA1 for another dataset in the "intersection of the two GO snapshots" as noted in the methods section. (But it is also not clear to me)
  
  
  ---
  
  