Answers.Body: <p>I broke down and finally scratched this itch. I have implemented a tool that performs read-group-aware text-pileup from a single SAM/BAM file. The tool is called <a href="https://pypi.org/project/streaming-pileupy/" rel="nofollow noreferrer">streaming_pileupy</a> and it's available from pypi for installation with pip. After installation the command would be:</p>
  <pre class="lang-bash prettyprint-override"><code>spileup input.bam sample_names.txt
  </code></pre>
  <p>The tool is pre-alpha and missing some features. Contributions are welcome.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-05T13:15:20.237Z
Answers.LastActivityDate: 2021-11-13T13:10:26.187Z
Answers.LastEditDate: 2021-11-13T13:10:26.187Z
Answers.UserDisplayName: winni2k
Body: <p>I would like to perform a <code>samtools mpileup</code> from a single file that contains thousands of read groups with different SM tags. I could split the bam by read group using <code>samtools split</code>, and then perform the mpileup, but splitting into thousands of files and then performing an mpileup from thousands of files is crazy slow on my machine.</p>
  
  <p>I could imagine an implementation using pysam, but I was hoping someone has a solution already.</p>
  
  <p>Bcftools mpileup already has this feature, but the feature was never implemented for the original mpileup format, which is what I need.</p>
  
  <h3>An example</h3>
  
  <p>Contents of <code>input.sam</code>:</p>
  
  <pre><code>@HD VN:1.6  SO:unknown
  @SQ SN:chr1 LN:1000000
  @RG ID:0    SM:sample_0
  @RG ID:1    SM:sample_1
  r0  0   chr1    24  0   1M  *   0   0   G   I   RG:Z:0
  r1  0   chr1    24  0   1M  *   0   0   G   I   RG:Z:1
  </code></pre>
  
  <p>When I run samtools mpileup I get this behavior:</p>
  
  <pre><code>&gt; samtools mpileup input.sam
  [mpileup] 2 samples in 1 input files
  chr1    24  N   2   ^!G$^!G$    II
  </code></pre>
  
  <p>Whereas, what I actually want is this output:</p>
  
  <pre><code>[mpileup] 2 samples in 1 input files
  chr1    24  N   1   ^!G$    I   1   ^!G$    I
  </code></pre>
  
  <p>For a long time I thought this was a bug in samtools mpileup, but it appears that it is a feature that was described/hinted at in the samtools documentation, but was never implemented: <a href="https://github.com/samtools/samtools/issues/599#issuecomment-604941884" rel="nofollow noreferrer">https://github.com/samtools/samtools/issues/599#issuecomment-604941884</a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-03T20:13:27.490Z
Comments.Text: been a long time since i use that.. you need the RG tag field the reads right? mpileup uses this to separate into different columns (i.e samples)
Comments.UserDisplayName: StupidWolf
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-05T07:01:51.760Z
Comments.Text: Won't GATK handle this? At the very least the old unified caller required input exactly like this.
Comments.UserDisplayName: Devon Ryan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-05T08:39:27.860Z
Comments.Text: That's a really good point! I did not realize gatk has a Pileup command. Unfortunately, that command (I tested v4.1.7.0) is also not read group aware :(
Comments.UserDisplayName: winni2k
Tags: <sam><samtools><pysam><mpileup>
Title: Is there a tool that can perform a read-group-aware mpileup from a single file?
ThreadText: # Is there a tool that can perform a read-group-aware mpileup from a single file?
  
  (Asked by: username_0 on 2020-05-03)
  
  I would like to perform a `samtools mpileup` from a single file that contains thousands of read groups with different SM tags. I could split the bam by read group using `samtools split`, and then perform the mpileup, but splitting into thousands of files and then performing an mpileup from thousands of files is crazy slow on my machine.
  
  I could imagine an implementation using pysam, but I was hoping someone has a solution already.
  
  Bcftools mpileup already has this feature, but the feature was never implemented for the original mpileup format, which is what I need.
  
  ### An example
  
  Contents of `input.sam`:
  
  ```
  @HD VN:1.6  SO:unknown
  @SQ SN:chr1 LN:1000000
  @RG ID:0    SM:sample_0
  @RG ID:1    SM:sample_1
  r0  0   chr1    24  0   1M  *   0   0   G   I   RG:Z:0
  r1  0   chr1    24  0   1M  *   0   0   G   I   RG:Z:1
  
  ```
  
  When I run samtools mpileup I get this behavior:
  
  ```
  > samtools mpileup input.sam
  [mpileup] 2 samples in 1 input files
  chr1    24  N   2   ^!G$^!G$    II
  
  ```
  
  Whereas, what I actually want is this output:
  
  ```
  [mpileup] 2 samples in 1 input files
  chr1    24  N   1   ^!G$    I   1   ^!G$    I
  
  ```
  
  For a long time I thought this was a bug in samtools mpileup, but it appears that it is a feature that was described/hinted at in the samtools documentation, but was never implemented: <https://github.com/samtools/samtools/issues/599#issuecomment-604941884>
  
  **Tags:** sam, samtools, pysam, mpileup
  
  > **Comments:**
  > **username_1**: been a long time since i use that.. you need the RG tag field the reads right? mpileup uses this to separate into different columns (i.e samples)  
  > **username_2**: Won't GATK handle this? At the very least the old unified caller required input exactly like this.  
  > **username_0**: That's a really good point! I did not realize gatk has a Pileup command. Unfortunately, that command (I tested v4.1.7.0) is also not read group aware :(  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2020-05-05. Score: 2 | Accepted answer
  
  I broke down and finally scratched this itch. I have implemented a tool that performs read-group-aware text-pileup from a single SAM/BAM file. The tool is called [streaming\_pileupy](https://pypi.org/project/streaming-pileupy/) and it's available from pypi for installation with pip. After installation the command would be:
  
  ```bash
  spileup input.bam sample_names.txt
  
  ```
  
  The tool is pre-alpha and missing some features. Contributions are welcome.
  
  
  ---
  
  