Answers.Body: <p>I am the author of the question / answer here <a href="https://bioinformatics.stackexchange.com/questions/2846/why-does-a-very-strong-blast-hit-get-lost-when-i-change-num-alignments-num-desc?noredirect=1&amp;lq=1">Why does a very strong BLAST hit get lost when I change num_alignments, num_descriptions or max_target_seqs parameter?</a></p>
  
  <blockquote>
    <p>To my knowledge changing the evalue threshold should no change the
    values themselves, it should just change the reporting.</p>
  </blockquote>
  
  <p>That is sadly not the case, though thats what I would say is typically expected.</p>
  
  <p>Yours was the exact situation that lead me to question the implementation of NCBI blast. I got results with an evalue above a certain value with a relaxed threshold, but with a stricter threshold (still lower than the value of the results I got) those results disappeared.</p>
  
  <p>From what I found all parameters imposing some threshold show this behavior and are applied in an early step of the program leading to prematurely removed results that later on might score higher than those thresholds. (Say your hit first has an evalue of 5: threshold 10 = ok and kept, threshold 1e-5 = gets removed, but that same hit gets improved further along in the program and then gets an evalue of 1e-6. Now this is only shown when you use an evalue threshold of 5 or higher, though 1e-6 is clearly also a result you would want to see when using a threshold of 3, 1, 0.01, ... etc.)</p>
  
  <p>I was relatively shocked by this and the fact that its <strong>basically nowhere officially documented and generally a not known fact</strong> what influence these options have.</p>
  
  <p>Sadly for now the stance of the developers is basically all is working as intended and the only feedback I got amounts to </p>
  
  <blockquote>
    <p>The safest approach is to relax (increase) the threshold and deal with a larger result set.</p>
  </blockquote>
  
  <p>aka Suck it up and deal with it.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-25T13:36:41.993Z
Answers.LastActivityDate: 2018-04-25T13:36:41.993Z
Answers.UserDisplayName: voiDnyx
Body: <p>I was teaching an introduction to bioinformatics when the students and I noticed strange Blast behavior that we couldn't explain.</p>
  
  <p>With the default <code>evalue</code> parameter, our best hit shows an evalue of <code>3e-06</code>, while that hit disappears with a evalue threshold of <code>1e-5</code> (which is bigger!)</p>
  
  <p><em>(after a few tries it seems that the evalue of the top hit starts changing with an evalue threshold of 2)</em></p>
  
  <p><strong>Input proteins:</strong></p>
  
  <pre><code>wget ftp://ftp.ncbi.nih.gov/refseq/B_taurus/mRNA_Prot/cow.1.protein.faa.gz
  wget ftp://ftp.ncbi.nih.gov/refseq/H_sapiens/mRNA_Prot/human.1.protein.faa.gz 
  gzip -d *.gz
  head -199 cow.1.protein.faa &gt; medium_cow.faa
  </code></pre>
  
  <p><strong>Database building:</strong></p>
  
  <pre><code>makeblastdb -in human.1.protein.faa -dbtype prot
  </code></pre>
  
  <p><strong>Blast command 1 (default):</strong></p>
  
  <pre><code>blastp -query medium_cow.faa -db human.1.protein.faa -outfmt 7 -out test_blast
  </code></pre>
  
  <p>output:</p>
  
  <pre><code># BLASTP 2.2.31+
  # Query: NP_001180091.1 SH2 domain-containing protein 1B [Bos taurus]
  # Database: human.1.protein.faa
  # Fields: query id, subject id, % identity, alignment length, mismatches, gap opens, q. start, q. end, s. start, s. end, evalue, bit score
  # 13 hits found
  NP_001180091.1  XP_011539315.1  29.41   102     61      3       5       100     144     240     3e-06   42.0
  </code></pre>
  
  <p><strong>Blast command 2 (-evalue 1e-05):</strong></p>
  
  <pre><code>blastp -query medium_cow.faa -db human.1.protein.faa -outfmt 7 -out test_blast -evalue 1e-05
  </code></pre>
  
  <p>output:</p>
  
  <pre><code># BLASTP 2.2.31+
  # Query: NP_001180091.1 SH2 domain-containing protein 1B [Bos taurus]
  # Database: human.1.protein.faa
  # 0 hits found
  </code></pre>
  
  <p><strong>Blast command 3 (-evalue 2):</strong></p>
  
  <pre><code>blastp -query medium_cow.faa -db human.1.protein.faa -outfmt 7 -out test_blast -evalue 2
  </code></pre>
  
  <p>output:</p>
  
  <pre><code># BLASTP 2.2.31+
  # Query: NP_001180091.1 SH2 domain-containing protein 1B [Bos taurus]
  # Database: human.1.protein.faa
  # Fields: query id, subject id, % identity, alignment length, mismatches, gap opens, q. start, q. end, s. start, s. end, evalue, bit score
  # 6 hits found
  NP_001180091.1  XP_005244948.1  40.62   32      18      1       4       35      394     424     0.076   29.3
  </code></pre>
  
  <p>I reproduced this behavior with Blast 2.2.31+, 2.7.1+ and the web blast at <a href="https://blast.ncbi.nlm.nih.gov" rel="noreferrer">https://blast.ncbi.nlm.nih.gov</a> (screenshots attached)</p>
  
  <p><a href="https://i.sstatic.net/RMVqa.png" rel="noreferrer"><img src="https://i.sstatic.net/RMVqa.png" alt="hits"></a></p>
  
  <p><a href="https://i.sstatic.net/GPxBG.png" rel="noreferrer"><img src="https://i.sstatic.net/GPxBG.png" alt="no hits"></a></p>
  
  <p>Anything obvious that my students and I are missing? To my knowledge changing the evalue threshold should no change the values themselves, it should just change the reporting.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-26T15:07:26.527Z
Comments.Text: Possible duplicate of [Why does a very strong BLAST hit get lost when I change num\_alignments, num\_descriptions or max\_target\_seqs parameter?](https://bioinformatics.stackexchange.com/questions/2846/why-does-a-very-strong-blast-hit-get-lost-when-i-change-num-alignments-num-desc). See the end of the answer to [this](https://bioinformatics.stackexchange.com/a/2847/48) question. e-value doesn't filter only, it influence the results.
Comments.UserDisplayName: llrs
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-26T16:34:34.323Z
Comments.Text: @Llopis that answer discusses parameters affecting the number of sequences to show. I don't see why the e-value would change here. Note that the actual e-value is different depending on the threshold set. The size of the DB, however, remains the same so it is very surprising that both the e-value and the bit-scores are different when using different e-value thresholds. These are two completely different hits, not the same hit shown or not shown depending on the options. Are you sure it's the same issue? What am I missing?
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-26T16:39:05.630Z
Comments.Text: That answers says that evalue parameter is not a threshold is an internal filter!! so using them alters the results and thus the sequences and the e-values: " -evalue
   -max_target_seqs
   -num_alignments
   -num_descriptions are NOT just filters applied to the results, but instead directly influence the results found. "
Comments.UserDisplayName: llrs
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-26T16:44:29.087Z
Comments.Text: Because the space searched is different it changes everything of the hits as far as I understood, not only if a sequence is a hit but also the e-value of the hits. But probably the best way to be sure it is a duplicate or not would be to contact them. (Don't cast your binding vote of duplicate if you aren't sure, perhaps it is not an exact duplicate but it is a highly related question)
Comments.UserDisplayName: llrs
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-26T16:58:42.597Z
Comments.Text: If this is just a case of sequences being filtered out, I think it is the same issue. The filter is applied to evalues calculated during the ungapped extension phase, so it's not surprising that they are different. If the actual reported evalue is different, that's another issue, but I don't see any evidence from the question that it is though (it would be nice to know what the evalue of the ```XP_005244948.1``` hit is in the first result)
Comments.UserDisplayName: heathobrien
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-26T17:54:50.803Z
Comments.Text: @Llopis (and heathobrien) argh I was looking at the query sequence's name, not the subject, so I saw different values. Duh!
Comments.UserDisplayName: terdon
Tags: <blast>
Title: Blast hits disappearing after changing -evalue
ThreadText: # Blast hits disappearing after changing -evalue
  
  (Asked by: username_0 on 2018-01-26)
  
  I was teaching an introduction to bioinformatics when the students and I noticed strange Blast behavior that we couldn't explain.
  
  With the default `evalue` parameter, our best hit shows an evalue of `3e-06`, while that hit disappears with a evalue threshold of `1e-5` (which is bigger!)
  
  *(after a few tries it seems that the evalue of the top hit starts changing with an evalue threshold of 2)*
  
  **Input proteins:**
  
  ```
  wget ftp://ftp.ncbi.nih.gov/refseq/B_taurus/mRNA_Prot/cow.1.protein.faa.gz
  wget ftp://ftp.ncbi.nih.gov/refseq/H_sapiens/mRNA_Prot/human.1.protein.faa.gz 
  gzip -d *.gz
  head -199 cow.1.protein.faa > medium_cow.faa
  
  ```
  
  **Database building:**
  
  ```
  makeblastdb -in human.1.protein.faa -dbtype prot
  
  ```
  
  **Blast command 1 (default):**
  
  ```
  blastp -query medium_cow.faa -db human.1.protein.faa -outfmt 7 -out test_blast
  
  ```
  
  output:
  
  ```
  # BLASTP 2.2.31+
  # Query: NP_001180091.1 SH2 domain-containing protein 1B [Bos taurus]
  # Database: human.1.protein.faa
  # Fields: query id, subject id, % identity, alignment length, mismatches, gap opens, q. start, q. end, s. start, s. end, evalue, bit score
  # 13 hits found
  NP_001180091.1  XP_011539315.1  29.41   102     61      3       5       100     144     240     3e-06   42.0
  
  ```
  
  **Blast command 2 (-evalue 1e-05):**
  
  ```
  blastp -query medium_cow.faa -db human.1.protein.faa -outfmt 7 -out test_blast -evalue 1e-05
  
  ```
  
  output:
  
  ```
  # BLASTP 2.2.31+
  # Query: NP_001180091.1 SH2 domain-containing protein 1B [Bos taurus]
  # Database: human.1.protein.faa
  # 0 hits found
  
  ```
  
  **Blast command 3 (-evalue 2):**
  
  ```
  blastp -query medium_cow.faa -db human.1.protein.faa -outfmt 7 -out test_blast -evalue 2
  
  ```
  
  output:
  
  ```
  # BLASTP 2.2.31+
  # Query: NP_001180091.1 SH2 domain-containing protein 1B [Bos taurus]
  # Database: human.1.protein.faa
  # Fields: query id, subject id, % identity, alignment length, mismatches, gap opens, q. start, q. end, s. start, s. end, evalue, bit score
  # 6 hits found
  NP_001180091.1  XP_005244948.1  40.62   32      18      1       4       35      394     424     0.076   29.3
  
  ```
  
  I reproduced this behavior with Blast 2.2.31+, 2.7.1+ and the web blast at <https://blast.ncbi.nlm.nih.gov> (screenshots attached)
  
  [![hits](https://i.sstatic.net/RMVqa.png)](https://i.sstatic.net/RMVqa.png)
  
  [![no hits](https://i.sstatic.net/GPxBG.png)](https://i.sstatic.net/GPxBG.png)
  
  Anything obvious that my students and I are missing? To my knowledge changing the evalue threshold should no change the values themselves, it should just change the reporting.
  
  **Tags:** blast
  
  > **Comments:**
  > **username_1**: Possible duplicate of [Why does a very strong BLAST hit get lost when I change num\_alignments, num\_descriptions or max\_target\_seqs parameter?](https://bioinformatics.stackexchange.com/questions/2846/why-does-a-very-strong-blast-hit-get-lost-when-i-change-num-alignments-num-desc). See the end of the answer to [this](https://bioinformatics.stackexchange.com/a/2847/48) question. e-value doesn't filter only, it influence the results.  
  > **username_2**: @Llopis that answer discusses parameters affecting the number of sequences to show. I don't see why the e-value would change here. Note that the actual e-value is different depending on the threshold set. The size of the DB, however, remains the same so it is very surprising that both the e-value and the bit-scores are different when using different e-value thresholds. These are two completely different hits, not the same hit shown or not shown depending on the options. Are you sure it's the same issue? What am I missing?  
  > **username_1**: That answers says that evalue parameter is not a threshold is an internal filter!! so using them alters the results and thus the sequences and the e-values: " -evalue  -max_target_seqs  -num_alignments  -num_descriptions are NOT just filters applied to the results, but instead directly influence the results found. "  
  > **username_1**: Because the space searched is different it changes everything of the hits as far as I understood, not only if a sequence is a hit but also the e-value of the hits. But probably the best way to be sure it is a duplicate or not would be to contact them. (Don't cast your binding vote of duplicate if you aren't sure, perhaps it is not an exact duplicate but it is a highly related question)  
  > **username_3**: If this is just a case of sequences being filtered out, I think it is the same issue. The filter is applied to evalues calculated during the ungapped extension phase, so it's not surprising that they are different. If the actual reported evalue is different, that's another issue, but I don't see any evidence from the question that it is though (it would be nice to know what the evalue of the ```XP_005244948.1``` hit is in the first result)  
  > **username_2**: @Llopis (and username_3) argh I was looking at the query sequence's name, not the subject, so I saw different values. Duh!  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_4 on 2018-04-25. Score: 3 | Accepted answer
  
  I am the author of the question / answer here [Why does a very strong BLAST hit get lost when I change num\_alignments, num\_descriptions or max\_target\_seqs parameter?](https://bioinformatics.stackexchange.com/questions/2846/why-does-a-very-strong-blast-hit-get-lost-when-i-change-num-alignments-num-desc?noredirect=1&lq=1)
  
  > To my knowledge changing the evalue threshold should no change the
  > values themselves, it should just change the reporting.
  
  That is sadly not the case, though thats what I would say is typically expected.
  
  Yours was the exact situation that lead me to question the implementation of NCBI blast. I got results with an evalue above a certain value with a relaxed threshold, but with a stricter threshold (still lower than the value of the results I got) those results disappeared.
  
  From what I found all parameters imposing some threshold show this behavior and are applied in an early step of the program leading to prematurely removed results that later on might score higher than those thresholds. (Say your hit first has an evalue of 5: threshold 10 = ok and kept, threshold 1e-5 = gets removed, but that same hit gets improved further along in the program and then gets an evalue of 1e-6. Now this is only shown when you use an evalue threshold of 5 or higher, though 1e-6 is clearly also a result you would want to see when using a threshold of 3, 1, 0.01, ... etc.)
  
  I was relatively shocked by this and the fact that its **basically nowhere officially documented and generally a not known fact** what influence these options have.
  
  Sadly for now the stance of the developers is basically all is working as intended and the only feedback I got amounts to
  
  > The safest approach is to relax (increase) the threshold and deal with a larger result set.
  
  aka Suck it up and deal with it.
  
  
  ---
  
  