Answers.Body: <p>Just to follow-up on the performance of <strong>minimap2</strong>: the solution @MaximillianPress described works. Thousands of hits are obtained for comparatively obscure genes (some will be duplicates). Parameterisation gives a 2-log<sub>10</sub> increase (astonishing).</p>
  <p>The issue is that its:</p>
  <ul>
  <li>relatively slow - precisely as described by @MP;</li>
  <li>very, very sensitive to parameterisation, just a single integer changes the output by a couple of thousand hits.</li>
  </ul>
  <p>I can see why there was concern about using it - such notable sensitive is an issue - and alternatively its clear why other investigators have had great success with it - hit the &quot;sweet-spot&quot; and its a bonanza. For my purposes it's fine - it's not a quantitation of complete genetic diversity, as long as I capture a reasonable proportion of diversity that is fine. Clearly, they'll be times the &quot;sweet-spot&quot; is missed but thats still okay (famous last words :-) ).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-03T12:56:50.487Z
Answers.LastActivityDate: 2023-10-03T16:09:08.267Z
Answers.LastEditDate: 2023-10-03T16:09:08.267Z
Answers.UserDisplayName: M__
Answers.Body: <h2>First of all, use short read option</h2>
  <p>The short read preset (<code>-x sr</code>) for minimap2 is heavily recommended for doing such mapping:</p>
  <p><code>minimap2 -x sr</code></p>
  <h2>Heng Li on minimap2 for related genomes</h2>
  <p>Heng Li (minimap2 author) has some suggestions on <a href="https://github.com/lh3/minimap2/issues/521" rel="nofollow noreferrer">a somewhat related github issue</a> (though it is now 4 years old):</p>
  <ol>
  <li>Minimap2 is not really intended for this purpose. For short reads he recommends <a href="https://github.com/uwb-linux/stampy" rel="nofollow noreferrer">stampy</a>, but it is clearly quite old (python2.7) and does not seem to have much recent developer history.</li>
  <li>Failing that, he suggests an option set such as follows:</li>
  </ol>
  <blockquote>
  <p>If you want to give minimap2 a try anyway, you can use the following setting:  <code>minimap2 -axsr -A2 -B4 -O4,24 -E2,1</code>
  If minimap2 is not sensitive enough, reduce -k and -w at the cost of mapping speed. Reducing -w should have relatively lower impact on speed, but reducing -k should be more sensitive.</p>
  </blockquote>
  <p>I would suggest engaging with Heng, he is quite responsive in general and may have more suggestions.</p>
  <h2>Other aligners for metagenomics</h2>
  <p>I would suggest that you look for more possibilities for alignment, or possibly <a href="https://www.google.com/url?sa=t&amp;rct=j&amp;q=&amp;esrc=s&amp;source=web&amp;cd=&amp;cad=rja&amp;uact=8&amp;ved=2ahUKEwjRub2x2NWBAxUXGDQIHShcD8kQFnoECBoQAQ&amp;url=https%3A%2F%2Fbmcbioinformatics.biomedcentral.com%2Farticles%2F10.1186%2Fs12859-022-05103-0&amp;usg=AOvVaw1qd9PVqzz2yA3iLh2KNjhk&amp;opi=89978449" rel="nofollow noreferrer">not using alignment in the first place</a>. However, this is very sensitive to your ultimate application, which I'm not sure that I understand from your post.</p>
  <p>I was pleased to see that <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9040843/" rel="nofollow noreferrer">others have reported good results using minimap2 for metagenomics</a>, which I was skeptical about, having been out of the field for a bit. Usually <a href="https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0105776" rel="nofollow noreferrer">in metagenomics there is an expected tradeoff</a> between sensitivity and specificity for the reasons that you state (this paper is old but it uses BLAST, so it is a sort of best case for sensitivity on unrelated genomes at the level of divergence that they test).</p>
  <p>There are of course new shotgun metagenomic aligners coming out all the time, <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9650795/" rel="nofollow noreferrer">here is one recent example</a> that I know nothing about but which may be interesting as a comparison.</p>
  <p>If you are really looking for a rich parameter set that allows low-stringency mappings, I would suggest going back to BLAST, though it will be much slower. minimap2 and related algorithms are to some extent dependent on the ability to find short seed matches, so there will necessarily be less than perfect sensitivity in your application.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-02T03:25:52.687Z
Answers.Text: Many thanks. Cool answer. I've addressed the issues you've below the final "-------"
Answers.UserDisplayName: M__
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-01T20:38:57.250Z
Answers.LastActivityDate: 2023-10-01T20:38:57.250Z
Answers.UserDisplayName: Maximilian Press
Body: <p>I'm leveraging <strong>minimap2</strong> to map select genes from short-read microbial fastq metagenomics zip files.</p>
  <pre><code>minimap2 --heap-sort=yes --splice-flank=no metagenome.fastq.gz fastaDB.fas 
  </code></pre>
  <p>What I noticed is the low tolerance of mismatch when I need to maximise mismatch because microbes have loads of genetic diversity. Thus the collection of references sequences are never really a reference, in fact the &quot;microbe reference genome&quot; is a misnomer, because <em>in situ</em> diversity of microbes is huge.</p>
  <p><strong>Question</strong> How do you relax the mapping stringency criteria for minimap2? If you are familiar with Blast parameters this would help alot here. I know blast but I've never parameterised it.</p>
  <p>The options are this are:</p>
  <p><em><a href="https://lh3.github.io/minimap2/minimap2.html" rel="nofollow noreferrer">Alignment options</a></em></p>
  <blockquote>
  <p>-A Matching score [2]</p>
  </blockquote>
  <blockquote>
  <p>-B INT    Mismatching penalty [4]</p>
  </blockquote>
  <blockquote>
  <p>-O INT1[,INT2] Gap open penalty [4,24].</p>
  </blockquote>
  <blockquote>
  <p>-s INT    Minimal peak DP alignment score to output [40].</p>
  </blockquote>
  <blockquote>
  <p>--score-N INT
  Score of a mismatch involving ambiguous bases <a href="https://lh3.github.io/minimap2/minimap2.html" rel="nofollow noreferrer">1</a></p>
  </blockquote>
  <p>Anything else that is relevant.</p>
  <p>Note, -f is a famous option, not sure it helps here.</p>
  <p>I'm going to code up a hyperparameterisation unit-test, but understanding the parameters first would really, really help.</p>
  <p>Alternative approaches welcome as well.</p>
  <hr />
  <p><em>Plugging minimap2</em>. I code high performance (hadoop-style, back-tracking, graph, traverse tall buildings in a stride  etc ...). The coder that wrote minimap has knowledge beyond, i.e. very fast run times directly from gzip file. I assume they are indexing then chunking the raw zip file then unzipping each 'chunk' JIT (just in time).</p>
  <hr />
  <p><strong>Thanks @MaximilianPress great answer</strong>. I'll fill in the backstory.</p>
  <p>I'm trapping total genetic diversity across Genbank and the metagenomes. My protocol to do both:</p>
  <ul>
  <li>Blasting out Genbank and then</li>
  <li>piping the output across metagenomes via in this case minimap2.</li>
  </ul>
  <p>Hence speed is key. So I ain't replacing Blast with minimap2 but leveraging minimap2 via the output of blast.</p>
  <p><em>However</em> on reflection of @MaximilianPress's discussion the answer is to generate a gold standard of diversity to assess alternative approaches.</p>
  
Tags: <fastq><blast><read-mapping><minimap2><parameters>
Title: Parameter optimisation of minimap2
ThreadText: # Parameter optimisation of minimap2
  
  (Asked by: username_0 on 2023-10-01)
  
  I'm leveraging **minimap2** to map select genes from short-read microbial fastq metagenomics zip files.
  
  ```
  minimap2 --heap-sort=yes --splice-flank=no metagenome.fastq.gz fastaDB.fas 
  
  ```
  
  What I noticed is the low tolerance of mismatch when I need to maximise mismatch because microbes have loads of genetic diversity. Thus the collection of references sequences are never really a reference, in fact the "microbe reference genome" is a misnomer, because *in situ* diversity of microbes is huge.
  
  **Question** How do you relax the mapping stringency criteria for minimap2? If you are familiar with Blast parameters this would help alot here. I know blast but I've never parameterised it.
  
  The options are this are:
  
  *[Alignment options](https://lh3.github.io/minimap2/minimap2.html)*
  
  > -A Matching score [2]
  
  > -B INT Mismatching penalty [4]
  
  > -O INT1[,INT2] Gap open penalty [4,24].
  
  > -s INT Minimal peak DP alignment score to output [40].
  
  > --score-N INT
  > Score of a mismatch involving ambiguous bases [1](https://lh3.github.io/minimap2/minimap2.html)
  
  Anything else that is relevant.
  
  Note, -f is a famous option, not sure it helps here.
  
  I'm going to code up a hyperparameterisation unit-test, but understanding the parameters first would really, really help.
  
  Alternative approaches welcome as well.
  
  ---
  
  *Plugging minimap2*. I code high performance (hadoop-style, back-tracking, graph, traverse tall buildings in a stride etc ...). The coder that wrote minimap has knowledge beyond, i.e. very fast run times directly from gzip file. I assume they are indexing then chunking the raw zip file then unzipping each 'chunk' JIT (just in time).
  
  ---
  
  **Thanks @username_1 great answer**. I'll fill in the backstory.
  
  I'm trapping total genetic diversity across Genbank and the metagenomes. My protocol to do both:
  
  * Blasting out Genbank and then
  * piping the output across metagenomes via in this case minimap2.
  
  Hence speed is key. So I ain't replacing Blast with minimap2 but leveraging minimap2 via the output of blast.
  
  *However* on reflection of @username_1's discussion the answer is to generate a gold standard of diversity to assess alternative approaches.
  
  **Tags:** fastq, blast, read-mapping, minimap2, parameters
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2023-10-01. Score: 2 | Accepted answer
  
  ## First of all, use short read option
  
  The short read preset (`-x sr`) for minimap2 is heavily recommended for doing such mapping:
  
  `minimap2 -x sr`
  
  ## Heng Li on minimap2 for related genomes
  
  Heng Li (minimap2 author) has some suggestions on [a somewhat related github issue](https://github.com/lh3/minimap2/issues/521) (though it is now 4 years old):
  
  1. Minimap2 is not really intended for this purpose. For short reads he recommends [stampy](https://github.com/uwb-linux/stampy), but it is clearly quite old (python2.7) and does not seem to have much recent developer history.
  2. Failing that, he suggests an option set such as follows:
  
  > If you want to give minimap2 a try anyway, you can use the following setting: `minimap2 -axsr -A2 -B4 -O4,24 -E2,1`
  > If minimap2 is not sensitive enough, reduce -k and -w at the cost of mapping speed. Reducing -w should have relatively lower impact on speed, but reducing -k should be more sensitive.
  
  I would suggest engaging with Heng, he is quite responsive in general and may have more suggestions.
  
  ## Other aligners for metagenomics
  
  I would suggest that you look for more possibilities for alignment, or possibly [not using alignment in the first place](https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwjRub2x2NWBAxUXGDQIHShcD8kQFnoECBoQAQ&url=https%3A%2F%2Fbmcbioinformatics.biomedcentral.com%2Farticles%2F10.1186%2Fs12859-022-05103-0&usg=AOvVaw1qd9PVqzz2yA3iLh2KNjhk&opi=89978449). However, this is very sensitive to your ultimate application, which I'm not sure that I understand from your post.
  
  I was pleased to see that [others have reported good results using minimap2 for metagenomics](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9040843/), which I was skeptical about, having been out of the field for a bit. Usually [in metagenomics there is an expected tradeoff](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0105776) between sensitivity and specificity for the reasons that you state (this paper is old but it uses BLAST, so it is a sort of best case for sensitivity on unrelated genomes at the level of divergence that they test).
  
  There are of course new shotgun metagenomic aligners coming out all the time, [here is one recent example](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9650795/) that I know nothing about but which may be interesting as a comparison.
  
  If you are really looking for a rich parameter set that allows low-stringency mappings, I would suggest going back to BLAST, though it will be much slower. minimap2 and related algorithms are to some extent dependent on the ability to find short seed matches, so there will necessarily be less than perfect sensitivity in your application.
  
  > **Comments:**
  > **username_0**: Many thanks. Cool answer. I've addressed the issues you've below the final "-------"  
  
  ---
  
  ### Answer by username_0 on 2023-10-03. Score: 1
  
  Just to follow-up on the performance of **minimap2**: the solution @MaximillianPress described works. Thousands of hits are obtained for comparatively obscure genes (some will be duplicates). Parameterisation gives a 2-log10 increase (astonishing).
  
  The issue is that its:
  
  * relatively slow - precisely as described by @MP;
  * very, very sensitive to parameterisation, just a single integer changes the output by a couple of thousand hits.
  
  I can see why there was concern about using it - such notable sensitive is an issue - and alternatively its clear why other investigators have had great success with it - hit the "sweet-spot" and its a bonanza. For my purposes it's fine - it's not a quantitation of complete genetic diversity, as long as I capture a reasonable proportion of diversity that is fine. Clearly, they'll be times the "sweet-spot" is missed but thats still okay (famous last words :-) ).
  
  
  ---
  
  