Answers.Body: <p>I used a local version of <code>segmasker</code> <strong>*</strong>. One way to solve this is,</p>
  <pre><code>perl -p -i -e 's/ //g' bioinfo.fa
  perl -p -e 's/\t/^/g' bioinfo.fa | segmasker | perl -p -e 's/^/\t/g'  &gt; bioinfo2.sm
  </code></pre>
  <p><em>Edit: I changed <code>,</code> for <code>^</code> because the OPs code already has <code>,</code> and will insert an extra tab (\t). Same change in the line below</em></p>
  <p>Here the input file you presented in the link is called <code>bioinfo.fa</code>. If you need the pipe explaining let me know, but it's
  fairly straight forward.</p>
  <p>If you want to preserve space characters, i.e. ' ' or &quot;\s&quot; - because in the code above I delete them - then (not tested):</p>
  <pre><code>perl -p -e 's/\t/^/g' bioinfo.fa | perl -p -e 's/ /_/g' | segmasker | perl -p -e 's/_/ /g' | perl -p -e 's/^/\t/g'  &gt; bioinfo2.sm
  </code></pre>
  <p>The issue the OP encounter is not unique to tabs (\t) but also occurs with spaces (\s), at least with <code>makebasedb</code>.</p>
  <p><strong>input</strong></p>
  <pre><code>&gt;A0A000 IPR004839   Aminotransferase, class I/classII   PF00155 41  381
  GVSAIDGMNTLVRGQKLPIFSSSGQPHSELAMQIARQASVPEEEEGGDDEEGSEFAVIFGAMGITQEEANEFMEDFERTGALERSVVFMNLADDPAVERTVTPRMVLTTAEYLAFEKDYHVLVILTDMTNYCEALREIGAAREEVPGRRGYPG
  &gt;A0A000 IPR010961   Tetrapyrrole biosynthesis, 5-aminolevulinic acid synthase   TIGR01821   12  391
  ...
  </code></pre>
  <p><strong>output</strong></p>
  <pre><code>&gt;A0A000 IPR004839       Aminotransferase        class_I/classII        PF00155 41      381
  41 - 53
  &gt;A0A000 IPR010961       Tetrapyrrole_biosynthesis       _5aminolevulinic_acid_synthase  TIGR01821       12      391
  41 - 53
  &gt;A0A000 IPR015421       Pyridoxal phosphate dependent transferase        major domainG3DSA:3.40.640.10  48      288
  106 - 126
  ...
  </code></pre>
  <p>Please don't ask about solving this via an upstream <code>makeblastdb</code> because that gets complicated. The rate limiting step, i.e. why it is ignoring tabs is because of <code>makeblastdb</code>. Thats a consequence of the way its written ... its a developer issue (see <strong>blurb</strong>)</p>
  <hr />
  <p><strong>*</strong> Local <code>segmasker</code> installation ...</p>
  <ol>
  <li>Install <code>conda</code> via <code>Anaconda</code> or <code>miniconda</code></li>
  <li><code>conda create -n blast3 python=3.11</code> (creating an environment ... yep 3.11 is out (wow!, if you don't understand why <strong>WOW!</strong> ask in chat)</li>
  <li><code>conda activate blast3</code> (this is the conda environment, you can call it anything)</li>
  <li><code>conda install -c bioconda blast</code></li>
  </ol>
  <p><code>perl | segmasker | perl</code> will now work. The Python installation is just good practice.</p>
  <hr />
  <p><strong>blurb about NCBI's blast</strong> The issue is <code>makeblastdb</code>, what happens is that long headers are a complete pain in algorithms because they use loads of RAM and you've got to carry them through reiterative loops (convergence etc ...). The way to solve it is via recoding (thats how I've done it). Essentially, you have a mini database forcing header recoding which activates at the start of the  algorithm and reassembles the original header at the end of the algorithm. The user never sees this step, but it's really important for efficiency. Thus within the reiteration the header isn't consuming RAM, because it's an internal code, beyond base 10 to squash the overhead even further. The thing is header can become a lot bigger than the sequence data and thats not cool when you've loads and loads of data, e.g. a blast database. NCBI just haven't bothered, like if you can't see it why do it attitude. What they've said is that 50 characters is the limitation of the header for <code>makeblastdb</code>... what NCBI is looking for is a unique id at the start of the header it then truncates everything beyond that, i.e. tabs, spaces ... anything it can grab. NCBI have carried this through into <code>segmasker</code> it's not needed, but <code>makeblastdb</code> and <code>segmasker</code> would normally be used together - thats why. In the OPs case this results in duplicate headers, so the file format falls over. This isn't the OPs fault.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-28T20:46:53.587Z
Answers.Text: Thank you for the exhaustive answer!
Answers.UserDisplayName: maciejwww
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-28T20:48:41.380Z
Answers.Text: I agree, **Python** 3.11 has many great features. What's your biggest WOW there?
Answers.UserDisplayName: maciejwww
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-29T01:06:46.380Z
Answers.Text: @maciejwww The criticism of Python is its slow. **3.11** is the major upgrade to focus on speed - its the fight back and may there be many more. Answer, you are welcome, note was a slight bug, I switched `,` for `^`.
Answers.UserDisplayName: M__
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-28T09:46:17.040Z
Answers.LastActivityDate: 2022-11-29T01:10:10.040Z
Answers.LastEditDate: 2022-11-29T01:10:10.040Z
Answers.UserDisplayName: M__
Body: <p>I try to pass a <em>.fasta</em> file with headers where tabs are used as separators as input file to <a href="https://toolshed.g2.bx.psu.edu/repository/display_tool?repository_id=1d92ebdf7e8d466c&amp;tool_config=%2Fsrv%2Ftoolshed-repos%2Fmain%2F000%2Frepo_241%2Ftools%2Fncbi_blast_plus%2Fncbi_segmasker_wrapper.xml&amp;changeset_revision=5edc472ec434" rel="nofollow noreferrer"><em>NCBI BLAST+ segmasker</em></a>:<br />
  <code>&gt;A0A059VAR9    IPR022879   V-type ATP synthase regulatory subunit B/beta</code></p>
  <p>Unfortunately, the <em>segmasker</em> output's headers are shortened to the first tab:<br />
  <code>&gt;A0A059VAR9</code></p>
  <p>Is there a way to get whole headers in the output file (without replacing all the tabs in the input file with white spaces)?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-26T12:17:19.633Z
Comments.Text: Could you supply a small data set please? You can fill is with dumbie amino acids ... I'll look at NCBI's`segmasker` local version because thats what I've got installed.
Comments.UserDisplayName: M__
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-26T12:26:44.487Z
Comments.Text: I think I also use local version. Here's a sample data: [link](https://pastebin.pl/view/ed83cade)
Comments.UserDisplayName: maciejwww
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-28T09:50:00.577Z
Comments.Text: Thanks for the file, answered below. Sorry about the delay. This question isn't simplistic.
Comments.UserDisplayName: M__
Tags: <fasta><phylogenetics><segmasker>
Title: How to get full headers with tabs as separators with segmasker?
ThreadText: # How to get full headers with tabs as separators with segmasker?
  
  (Asked by: username_0 on 2022-10-24)
  
  I try to pass a *.fasta* file with headers where tabs are used as separators as input file to [*NCBI BLAST+ segmasker*](https://toolshed.g2.bx.psu.edu/repository/display_tool?repository_id=1d92ebdf7e8d466c&tool_config=%2Fsrv%2Ftoolshed-repos%2Fmain%2F000%2Frepo_241%2Ftools%2Fncbi_blast_plus%2Fncbi_segmasker_wrapper.xml&changeset_revision=5edc472ec434):  
  `>A0A059VAR9 IPR022879 V-type ATP synthase regulatory subunit B/beta`
  
  Unfortunately, the *segmasker* output's headers are shortened to the first tab:  
  `>A0A059VAR9`
  
  Is there a way to get whole headers in the output file (without replacing all the tabs in the input file with white spaces)?
  
  **Tags:** fasta, phylogenetics, segmasker
  
  > **Comments:**
  > **username_1**: Could you supply a small data set please? You can fill is with dumbie amino acids ... I'll look at NCBI's`segmasker` local version because thats what I've got installed.  
  > **username_0**: I think I also use local version. Here's a sample data: [link](https://pastebin.pl/view/ed83cade)  
  > **username_1**: Thanks for the file, answered below. Sorry about the delay. This question isn't simplistic.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-11-28. Score: 1 | Accepted answer
  
  I used a local version of `segmasker` **\***. One way to solve this is,
  
  ```
  perl -p -i -e 's/ //g' bioinfo.fa
  perl -p -e 's/\t/^/g' bioinfo.fa | segmasker | perl -p -e 's/^/\t/g'  > bioinfo2.sm
  
  ```
  
  *Edit: I changed `,` for `^` because the OPs code already has `,` and will insert an extra tab (\t). Same change in the line below*
  
  Here the input file you presented in the link is called `bioinfo.fa`. If you need the pipe explaining let me know, but it's
  fairly straight forward.
  
  If you want to preserve space characters, i.e. ' ' or "\s" - because in the code above I delete them - then (not tested):
  
  ```
  perl -p -e 's/\t/^/g' bioinfo.fa | perl -p -e 's/ /_/g' | segmasker | perl -p -e 's/_/ /g' | perl -p -e 's/^/\t/g'  > bioinfo2.sm
  
  ```
  
  The issue the OP encounter is not unique to tabs (\t) but also occurs with spaces (\s), at least with `makebasedb`.
  
  **input**
  
  ```
  >A0A000 IPR004839   Aminotransferase, class I/classII   PF00155 41  381
  GVSAIDGMNTLVRGQKLPIFSSSGQPHSELAMQIARQASVPEEEEGGDDEEGSEFAVIFGAMGITQEEANEFMEDFERTGALERSVVFMNLADDPAVERTVTPRMVLTTAEYLAFEKDYHVLVILTDMTNYCEALREIGAAREEVPGRRGYPG
  >A0A000 IPR010961   Tetrapyrrole biosynthesis, 5-aminolevulinic acid synthase   TIGR01821   12  391
  ...
  
  ```
  
  **output**
  
  ```
  >A0A000 IPR004839       Aminotransferase        class_I/classII        PF00155 41      381
  41 - 53
  >A0A000 IPR010961       Tetrapyrrole_biosynthesis       _5aminolevulinic_acid_synthase  TIGR01821       12      391
  41 - 53
  >A0A000 IPR015421       Pyridoxal phosphate dependent transferase        major domainG3DSA:3.40.640.10  48      288
  106 - 126
  ...
  
  ```
  
  Please don't ask about solving this via an upstream `makeblastdb` because that gets complicated. The rate limiting step, i.e. why it is ignoring tabs is because of `makeblastdb`. Thats a consequence of the way its written ... its a developer issue (see **blurb**)
  
  ---
  
  **\*** Local `segmasker` installation ...
  
  1. Install `conda` via `Anaconda` or `miniconda`
  2. `conda create -n blast3 python=3.11` (creating an environment ... yep 3.11 is out (wow!, if you don't understand why **WOW!** ask in chat)
  3. `conda activate blast3` (this is the conda environment, you can call it anything)
  4. `conda install -c bioconda blast`
  
  `perl | segmasker | perl` will now work. The Python installation is just good practice.
  
  ---
  
  **blurb about NCBI's blast** The issue is `makeblastdb`, what happens is that long headers are a complete pain in algorithms because they use loads of RAM and you've got to carry them through reiterative loops (convergence etc ...). The way to solve it is via recoding (thats how I've done it). Essentially, you have a mini database forcing header recoding which activates at the start of the algorithm and reassembles the original header at the end of the algorithm. The user never sees this step, but it's really important for efficiency. Thus within the reiteration the header isn't consuming RAM, because it's an internal code, beyond base 10 to squash the overhead even further. The thing is header can become a lot bigger than the sequence data and thats not cool when you've loads and loads of data, e.g. a blast database. NCBI just haven't bothered, like if you can't see it why do it attitude. What they've said is that 50 characters is the limitation of the header for `makeblastdb`... what NCBI is looking for is a unique id at the start of the header it then truncates everything beyond that, i.e. tabs, spaces ... anything it can grab. NCBI have carried this through into `segmasker` it's not needed, but `makeblastdb` and `segmasker` would normally be used together - thats why. In the OPs case this results in duplicate headers, so the file format falls over. This isn't the OPs fault.
  
  > **Comments:**
  > **username_0**: Thank you for the exhaustive answer!  
  > **username_0**: I agree, **Python** 3.11 has many great features. What's your biggest WOW there?  
  > **username_1**: @username_0 The criticism of Python is its slow. **3.11** is the major upgrade to focus on speed - its the fight back and may there be many more. Answer, you are welcome, note was a slight bug, I switched `,` for `^`.  
  
  ---
  
  