Answers.Body: <p>The evalue associated with a blast HSP (High-scoring Segment Pair) is a measure of how often you would expect to find such an HSP in a database of this size by pure chance. For instance, if you blast the sequence <code>ACTG</code> against the entirety of the NCBI's <code>nr</code> database, you would expect several hundred thousand hits by pure chance. That is, several hundred thousand sequences that can be aligned to your query but are in no way related to it (not actual homologs). Those hits don't actually mean anything, so you use the evalue to limit the results to those that are unlikely to be found by chance and, therefore, more likely to have biological meaning if hits are found. </p>
  
  <p>So, setting the evalue to 200000 means you are interested even in HSPs whose characteristics (score, length, etc) mean you would expect to find &lt;=200000 HSPs with such characteristics by pure chance. This evalue is pointless. </p>
  
  <p>Because your e-value is so high, BLAST will report absolutely anything and everything it finds, and the vast majority of your results will be meaningless and irrelevant. If you want a permissive e-value, you could use <code>1</code> or even <code>10</code> but 200000 is way too much. You usually only want negative e-values, cases where you wouldn't expect to find even one such HSP by chance. For highly dissimilar sequences, I would recommend you use 0.1 or something of that scale. </p>
  
  <p>Changing the evalue will speed things up considerably. That said, you should always expect different times for different queries since the time it takes to find hits will always depend on the number of hits that can be found. If your sequence can be aligned to many of the sequences in the database, it will take longer to fetch all results. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-10T09:23:08.983Z
Answers.LastActivityDate: 2017-08-10T10:05:48.027Z
Answers.LastEditDate: 2017-08-10T10:05:48.027Z
Answers.UserDisplayName: terdon
Answers.Body: <p>Try changing you <code>evalue</code> threshold to something like <code>0.001</code>.</p>
  
  <p>If you blasted a sequence against a database containing 10,000 sequences and got a hit with an evalue of <code>1</code>, this would mean that, given the size of your database and length of protein, you would expect to find a match as good as (or better than) the hit returned once purely by chance. If the <code>evalue</code> threshold in your query was set to <code>0.01</code> (for example) then this hit would not be returned.</p>
  
  <p>If you got a hit with an <code>evalue</code> of 200,000 this would mean you would expect to see 200,000 matches with a similar score simply by chance. If your <code>evalue</code> threshold were set to 200000, then all of the hits with an evalue &lt;=200000 would be considered significant. If you set it to <code>0.001</code> then only those hits with <code>evalues</code> &lt;= <code>0.001</code> would be considered significant. </p>
  
  <p>This means that (depending on the size of the data base) almost every sequence in your query set will be considered as significant given your threshold, which will slow down your search hugely. </p>
  
  <p>See the the section on E values <a href="https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&amp;PAGE_TYPE=BlastDocs&amp;DOC_TYPE=FAQ#expect" rel="nofollow noreferrer">here</a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-10T08:35:02.367Z
Answers.LastActivityDate: 2017-08-10T10:12:11.917Z
Answers.LastEditDate: 2017-08-10T10:12:11.917Z
Answers.UserDisplayName: fugu
Body: <p>I'm running blast+ searches for a list of proteins using specific parameters.</p>
  
  <p>This is my search</p>
  
  <pre><code> blastp -query protein.fasta -remote -db refseq_protein -out 
  protein_refseq_blast.txt -outfmt '6 qseqid sseqid qstart qend length evalue' -
  entrez_query 'txid2 [ORGN] or txid4751 [ORGN]' -evalue 200000 -max_target_seqs 10000
  </code></pre>
  
  <p>For some proteins, this runs moderately quickly, ~5-10 mins. For other proteins, this command never stops running, even after an hour or so. I'm wondering if this variability in speed is due to any part of this choice in parameters, is simply variability in speed for the BLAST servers. Is it common for BLAST searches to take this long? Is there a way to run all of them in parallel using these parameters if this is the case?</p>
  
  <p>Thanks.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-08-10T08:16:52.583Z
Comments.Text: FWIW, I remember experiencing very long waiting times when running remote blast from Biopython.
Comments.UserDisplayName: bli
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-08-10T08:18:21.173Z
Comments.Text: You have your `evalue` set incredibly high `-evalue 200000`, which might affect the time it takes to return a query.
Comments.UserDisplayName: fugu
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-08-10T08:24:03.290Z
Comments.Text: The variability could be simply because the proteins are very different sizes but even 5-10 mins is very long for a single protein, as fugu says, your evalue is crazy! Probably reducing the evalue to a 'normal value' (<1) will resolve this
Comments.UserDisplayName: Chris_Rands
Tags: <blast>
Title: Major variability in speed of BLAST between proteins
ThreadText: # Major variability in speed of BLAST between proteins
  
  (Asked by: username_0 on 2017-08-09)
  
  I'm running blast+ searches for a list of proteins using specific parameters.
  
  This is my search
  
  ```
   blastp -query protein.fasta -remote -db refseq_protein -out 
  protein_refseq_blast.txt -outfmt '6 qseqid sseqid qstart qend length evalue' -
  entrez_query 'txid2 [ORGN] or txid4751 [ORGN]' -evalue 200000 -max_target_seqs 10000
  
  ```
  
  For some proteins, this runs moderately quickly, ~5-10 mins. For other proteins, this command never stops running, even after an hour or so. I'm wondering if this variability in speed is due to any part of this choice in parameters, is simply variability in speed for the BLAST servers. Is it common for BLAST searches to take this long? Is there a way to run all of them in parallel using these parameters if this is the case?
  
  Thanks.
  
  **Tags:** blast
  
  > **Comments:**
  > **username_1**: FWIW, I remember experiencing very long waiting times when running remote blast from Biopython.  
  > **username_2**: You have your `evalue` set incredibly high `-evalue 200000`, which might affect the time it takes to return a query.  
  > **username_3**: The variability could be simply because the proteins are very different sizes but even 5-10 mins is very long for a single protein, as username_2 says, your evalue is crazy! Probably reducing the evalue to a 'normal value' (<1) will resolve this  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2017-08-10. Score: 5 | Accepted answer
  
  Try changing you `evalue` threshold to something like `0.001`.
  
  If you blasted a sequence against a database containing 10,000 sequences and got a hit with an evalue of `1`, this would mean that, given the size of your database and length of protein, you would expect to find a match as good as (or better than) the hit returned once purely by chance. If the `evalue` threshold in your query was set to `0.01` (for example) then this hit would not be returned.
  
  If you got a hit with an `evalue` of 200,000 this would mean you would expect to see 200,000 matches with a similar score simply by chance. If your `evalue` threshold were set to 200000, then all of the hits with an evalue <=200000 would be considered significant. If you set it to `0.001` then only those hits with `evalues` <= `0.001` would be considered significant.
  
  This means that (depending on the size of the data base) almost every sequence in your query set will be considered as significant given your threshold, which will slow down your search hugely.
  
  See the the section on E values [here](https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&PAGE_TYPE=BlastDocs&DOC_TYPE=FAQ#expect)
  
  
  ---
  
  ### Answer by username_4 on 2017-08-10. Score: 3
  
  The evalue associated with a blast HSP (High-scoring Segment Pair) is a measure of how often you would expect to find such an HSP in a database of this size by pure chance. For instance, if you blast the sequence `ACTG` against the entirety of the NCBI's `nr` database, you would expect several hundred thousand hits by pure chance. That is, several hundred thousand sequences that can be aligned to your query but are in no way related to it (not actual homologs). Those hits don't actually mean anything, so you use the evalue to limit the results to those that are unlikely to be found by chance and, therefore, more likely to have biological meaning if hits are found.
  
  So, setting the evalue to 200000 means you are interested even in HSPs whose characteristics (score, length, etc) mean you would expect to find <=200000 HSPs with such characteristics by pure chance. This evalue is pointless.
  
  Because your e-value is so high, BLAST will report absolutely anything and everything it finds, and the vast majority of your results will be meaningless and irrelevant. If you want a permissive e-value, you could use `1` or even `10` but 200000 is way too much. You usually only want negative e-values, cases where you wouldn't expect to find even one such HSP by chance. For highly dissimilar sequences, I would recommend you use 0.1 or something of that scale.
  
  Changing the evalue will speed things up considerably. That said, you should always expect different times for different queries since the time it takes to find hits will always depend on the number of hits that can be found. If your sequence can be aligned to many of the sequences in the database, it will take longer to fetch all results.
  
  
  ---
  
  