Answers.Body: <p>The problem I had was solved by <code>num_alignments</code>, which is more complete than the first section.  I do not think that the text truncation is solvable.  However, parsing of the latter half solved the problem.</p>
  <p>the correct way to get the most complete output is:</p>
  <p><code>time ~/bin/ncbi-blast-2.14.0+-src/c++/ReleaseMT/bin/blastp -db nr -query X.fasta -out nr.X.0 -outfmt 0 -num_descriptions 4000 -remote -num_alignments 4000</code></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-19T19:28:41.957Z
Answers.LastActivityDate: 2023-06-20T22:13:22.877Z
Answers.LastEditDate: 2023-06-20T22:13:22.877Z
Answers.UserDisplayName: con
Answers.Body: <p>Hmmm, firstly can you attempt multi-threading on a remote command via <code> -num_threads 8</code>...</p>
  <pre><code>blastp -db nr -query X.fasta -out nr.X -num_threads 8 -num_descriptions 4000 -remote
  </code></pre>
  <p>It's a worth a try. Otherwise it could likely be a function of the network access or remote server resources. I don't think this is a function of the local RAM. Therefore performing a remote blast to download relevant sections of the NCBI database, convert that into a local database and then perform local blast seems the best strategy.</p>
  <p>Multi-threading and <code>-outfmt</code> work well with local blast.</p>
  <pre><code>blastp -query query.fa -db test.db -outfmt 6 -max_target_seqs 1000 -num_threads 8 -out blasthits.txt
  </code></pre>
  <p>The alignment is a function of the output formats in the above example its <code>-outfmt 6</code> that is not useful, but <code>1</code> to <code>4</code> are relevant as follows.</p>
  <pre><code> -outfmt &lt;String&gt;
     alignment view options:
       0 = pairwise,
       1 = query-anchored showing identities,
       2 = query-anchored no identities,
       3 = flat query-anchored, show identities,
       4 = flat query-anchored, no identities,
       5 = XML Blast output,
       6 = tabular,
       7 = tabular with comment lines,
       8 = Text ASN.1,
       9 = Binary ASN.1,
      10 = Comma-separated values,
      11 = BLAST archive format (ASN.1) 
  </code></pre>
  <p>Multi-threading/parallelisation is the way to overcome memory issues, but really effective only in local operations not remote operations. Again, I think there's too much traffic/ or too many resources requested on NCBI server - and thats where the memory error is occurring.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-17T01:34:57.610Z
Answers.Text: There's no RAM issue, NCBI said they would deprioritize me. I just need to get all of the alignments showing in the 2nd part of the output file, and I'm not sure how to write the command that correctly. So far, option 0 is best, but I've yet to find a way of telling blastp to not truncate the output, and output the alignments for all
Answers.UserDisplayName: con
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-17T02:30:52.773Z
Answers.Text: I did say I thought local RAM was unlikely to be RAM. *NCBI have deprioritised you?* **Wow**.  Biopython does use scheduling so maybe their blast package may be better. `-outfmt 1` is probably the best.  I've never encounter truncation in blast so dunno. What I'm recommending is a two stage process and perform the blast locally.
Answers.UserDisplayName: M__
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-16T23:09:55.073Z
Answers.LastActivityDate: 2023-06-16T23:19:12.110Z
Answers.LastEditDate: 2023-06-16T23:19:12.110Z
Answers.UserDisplayName: M__
Body: <p>I am running a blastp search:</p>
  <pre><code>time ~/bin/ncbi-blast-2.14.0+-src/c++/ReleaseMT/bin/blastp -db nr -query X.fasta -out nr.X -num_descriptions 4000 -remote
  </code></pre>
  <p>but I have 2 problems with the output:</p>
  <p>I. Under <code>Sequences producing significant alignments:</code>,
  the output is truncated, and I can't read the exact species name:</p>
  <pre><code>XP_002906761.1 elongation factor 3, putative [Phytophthora infest...  848     0.0  
  KAG3108573.1 Elongation factor 3 [Phytophthora idaei] &gt;gb|KAG3151...  845     0.0  
  KAG2772535.1 Elongation factor 3 [Phytophthora cactorum] &gt;gb|KAG2...  845     0.0  
  KAG6976665.1 hypothetical protein JG688_00001147 [Phytophthora al...  844     0.0
  </code></pre>
  <p>II.</p>
  <p>The above sequences do not show up on the detailed sequence alignments below.</p>
  <p>I have tried numerous variations of &quot;outfmt&quot; variations, but now am getting warnings that I'm using the system too much.  So far, output format <code>0</code> is the best.</p>
  <p>How can I get the full names and not the truncations?</p>
  <p>Why aren't the above species showing up later on, and how can I get the full alignment to show?</p>
  
Tags: <blast><blastp>
Title: How to get full alignment output from BLASTP?
ThreadText: # How to get full alignment output from BLASTP?
  
  (Asked by: username_0 on 2023-06-16)
  
  I am running a blastp search:
  
  ```
  time ~/bin/ncbi-blast-2.14.0+-src/c++/ReleaseMT/bin/blastp -db nr -query X.fasta -out nr.X -num_descriptions 4000 -remote
  
  ```
  
  but I have 2 problems with the output:
  
  I. Under `Sequences producing significant alignments:`,
  the output is truncated, and I can't read the exact species name:
  
  ```
  XP_002906761.1 elongation factor 3, putative [Phytophthora infest...  848     0.0  
  KAG3108573.1 Elongation factor 3 [Phytophthora idaei] >gb|KAG3151...  845     0.0  
  KAG2772535.1 Elongation factor 3 [Phytophthora cactorum] >gb|KAG2...  845     0.0  
  KAG6976665.1 hypothetical protein JG688_00001147 [Phytophthora al...  844     0.0
  
  ```
  
  II.
  
  The above sequences do not show up on the detailed sequence alignments below.
  
  I have tried numerous variations of "outfmt" variations, but now am getting warnings that I'm using the system too much. So far, output format `0` is the best.
  
  How can I get the full names and not the truncations?
  
  Why aren't the above species showing up later on, and how can I get the full alignment to show?
  
  **Tags:** blast, blastp
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_0 on 2023-06-19. Score: -2 | Accepted answer
  
  The problem I had was solved by `num_alignments`, which is more complete than the first section. I do not think that the text truncation is solvable. However, parsing of the latter half solved the problem.
  
  the correct way to get the most complete output is:
  
  `time ~/bin/ncbi-blast-2.14.0+-src/c++/ReleaseMT/bin/blastp -db nr -query X.fasta -out nr.X.0 -outfmt 0 -num_descriptions 4000 -remote -num_alignments 4000`
  
  
  ---
  
  ### Answer by username_1 on 2023-06-16. Score: 1
  
  Hmmm, firstly can you attempt multi-threading on a remote command via  `-num_threads 8`...
  
  ```
  blastp -db nr -query X.fasta -out nr.X -num_threads 8 -num_descriptions 4000 -remote
  
  ```
  
  It's a worth a try. Otherwise it could likely be a function of the network access or remote server resources. I don't think this is a function of the local RAM. Therefore performing a remote blast to download relevant sections of the NCBI database, convert that into a local database and then perform local blast seems the best strategy.
  
  Multi-threading and `-outfmt` work well with local blast.
  
  ```
  blastp -query query.fa -db test.db -outfmt 6 -max_target_seqs 1000 -num_threads 8 -out blasthits.txt
  
  ```
  
  The alignment is a function of the output formats in the above example its `-outfmt 6` that is not useful, but `1` to `4` are relevant as follows.
  
  ```
   -outfmt <String>
     alignment view options:
       0 = pairwise,
       1 = query-anchored showing identities,
       2 = query-anchored no identities,
       3 = flat query-anchored, show identities,
       4 = flat query-anchored, no identities,
       5 = XML Blast output,
       6 = tabular,
       7 = tabular with comment lines,
       8 = Text ASN.1,
       9 = Binary ASN.1,
      10 = Comma-separated values,
      11 = BLAST archive format (ASN.1) 
  
  ```
  
  Multi-threading/parallelisation is the way to overcome memory issues, but really effective only in local operations not remote operations. Again, I think there's too much traffic/ or too many resources requested on NCBI server - and thats where the memory error is occurring.
  
  > **Comments:**
  > **username_0**: There's no RAM issue, NCBI said they would deprioritize me. I just need to get all of the alignments showing in the 2nd part of the output file, and I'm not sure how to write the command that correctly. So far, option 0 is best, but I've yet to find a way of telling blastp to not truncate the output, and output the alignments for all  
  > **username_1**: I did say I thought local RAM was unlikely to be RAM. *NCBI have deprioritised you?* **Wow**.  Biopython does use scheduling so maybe their blast package may be better. `-outfmt 1` is probably the best.  I've never encounter truncation in blast so dunno. What I'm recommending is a two stage process and perform the blast locally.  
  
  ---
  
  