Answers.Body: <p>I think allowing <code>samtools fixmate</code> to use additional threads is unlikely to provide any performance benefit given that <code>-u</code> (for uncompressed output) eliminates the need for compression/decompression. I suspect that the main computational effort for <code>samtools fixmate</code> (i.e. fixing mate-pair information) will remain single-threaded. That said, the best way to verify any and all performance assumptions is to test and profile the pipeline. In my experience though, specifying <code>-@</code> (for additional threads) is most beneficial when used with <code>samtools sort</code> where multi-threading can significantly improve performance. You may not even notice the impact of additional decompression or compression threads at the beginning and end of the pipeline.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-13T11:56:30.257Z
Answers.LastActivityDate: 2024-12-13T11:56:30.257Z
Answers.UserDisplayName: Steve
Body: <p>Samtools' <a href="http://www.htslib.org/algorithms/duplicate.html" rel="nofollow noreferrer">documentation regarding duplicate marking</a> suggests the following pipeline:</p>
  <pre><code>samtools collate -@ 4 -O -u example.bam | samtools fixmate -@ 4 -m -u - - | samtools sort -@ 4 -u - | samtools markdup -@ 4 - markdup.bam
  </code></pre>
  <p>With the following comment:</p>
  <blockquote>
  <p>The -u option produces uncompressed output. Since the output is not being written to disk then size is unimportant and avoiding compress/decompress reduces the amount of work that needs doing. The -@ option turns on threading which can speed things up.</p>
  </blockquote>
  <p>However, the documentation of <code>samtools fixmate</code> (*) says:</p>
  <blockquote>
  <p>-@, --threads INT</p>
  <pre><code>Number of input/output compression threads to use in addition to main thread [0]. 
  </code></pre>
  </blockquote>
  <p>Since option <code>-u</code> is used upstream of fixmate and since fixmate also uses option <code>-u</code>, I expect fixmate to have no &quot;input/output compression&quot; to perform at all. <strong>Is providing threads to fixmate going to be of any help under these circumstances?</strong></p>
  <p>(*) <a href="http://www.htslib.org/doc/samtools-fixmate.html" rel="nofollow noreferrer">http://www.htslib.org/doc/samtools-fixmate.html</a></p>
  <p>(<code>samtools collate</code> will read compressed input, <code>samtools sort</code> also uses threads for sorting, and <code>samtools markdup</code> will compress its output, therefore I expect some effect for option <code>-@, --threads</code> there.)</p>
  
Tags: <sam><samtools><multithreading>
Title: samtools fixmate threads for uncompressed bam data in pipeline
ThreadText: # samtools fixmate threads for uncompressed bam data in pipeline
  
  (Asked by: username_0 on 2024-12-12)
  
  Samtools' [documentation regarding duplicate marking](http://www.htslib.org/algorithms/duplicate.html) suggests the following pipeline:
  
  ```
  samtools collate -@ 4 -O -u example.bam | samtools fixmate -@ 4 -m -u - - | samtools sort -@ 4 -u - | samtools markdup -@ 4 - markdup.bam
  
  ```
  
  With the following comment:
  
  > The -u option produces uncompressed output. Since the output is not being written to disk then size is unimportant and avoiding compress/decompress reduces the amount of work that needs doing. The -@ option turns on threading which can speed things up.
  
  However, the documentation of `samtools fixmate` (\*) says:
  
  > -@, --threads INT
  >
  > ```
  > Number of input/output compression threads to use in addition to main thread [0]. 
  >
  > ```
  
  Since option `-u` is used upstream of fixmate and since fixmate also uses option `-u`, I expect fixmate to have no "input/output compression" to perform at all. **Is providing threads to fixmate going to be of any help under these circumstances?**
  
  (\*) <http://www.htslib.org/doc/samtools-fixmate.html>
  
  (`samtools collate` will read compressed input, `samtools sort` also uses threads for sorting, and `samtools markdup` will compress its output, therefore I expect some effect for option `-@, --threads` there.)
  
  **Tags:** sam, samtools, multithreading
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-12-13. Score: 3
  
  I think allowing `samtools fixmate` to use additional threads is unlikely to provide any performance benefit given that `-u` (for uncompressed output) eliminates the need for compression/decompression. I suspect that the main computational effort for `samtools fixmate` (i.e. fixing mate-pair information) will remain single-threaded. That said, the best way to verify any and all performance assumptions is to test and profile the pipeline. In my experience though, specifying `-@` (for additional threads) is most beneficial when used with `samtools sort` where multi-threading can significantly improve performance. You may not even notice the impact of additional decompression or compression threads at the beginning and end of the pipeline.
  
  
  ---
  
  