Answers.Body: <p>As has been said before, mappability to the 'human genome' depends on a number of factors, among these the reference version and type of reads, for which you are interested in GRCh38 and 2x150bp reads. Although I am not aware of numbers accounting for these particular reference and type of Illumina reads, the 1000 genomes project has provided the community with a similar and close estimation that you might be interested in considering regarding your inquiry.</p>
  
  <p>Similar to your question, the 1K project estimated 'the proportion of the human genome that is less accessible to short reads'. In these estimates the human genome is GRCh37 and the types of reads in question are mostly 2x Illumina with a mixture of lengths with the longest being up to 250bp. In these estimates each base in the human genome is considered (and marked) 'less accesible' according to these criteria:  </p>
  
  <p>L - depth of coverage is much lower than average</p>
  
  <p>H - depth of coverage is much higher than average</p>
  
  <p>Z - too many reads with zero mapping quality overlap this position</p>
  
  <p>Q - the average mapping quality at the position is too low</p>
  
  <p>Each of these criteria has "standard" and "strict" thresholds for a base to be considered - or not - in each category. You can read more in the link below: </p>
  
  <p><a href="ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/accessible_genome_masks/README.accessible_genome_mask.20140520" rel="noreferrer">ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/accessible_genome_masks/README.accessible_genome_mask.20140520</a></p>
  
  <p>According to the strict thresholds, the human genome has about 16.8% of "Z" and 3.1% of "Q" bases, respectively. Considering the "Z" and "Q" criteria as a proxy for ~ mapq=0, about 19.9% of the human genome can not be uniquely mapped. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-20T14:51:07.630Z
Answers.LastActivityDate: 2017-07-20T14:51:07.630Z
Answers.UserDisplayName: JRodrigoF
Answers.Body: <p>Mapping quality is determined by the repetitiveness of the genome, the sequencing error rate, insert size, the capability of the mapper and the nasty heuristics behind the mapper. MAPQ=0 to one mapper is not necessarily MAPQ=0 to another.</p>
  
  <p>That said, I get what you mean. You want to know the uniqueness/repetitiveness. It is still hard if you want to get a useful answer. For 150bp reads, each reference position is covered by 150 reads. What if 50 of them have no other exact hits elsewhere, but the rest 100 have? Is this position a repeat or not? In addition, what if the 50 are unique only because one mismatch? If there is a variant at that mismatch, the 50 would become repeats or mapped elsewhere.</p>
  
  <p>My preference is to say a position is "unique" under k-long reads if over k/2 reads overlapping the position have no other perfect or 1-mismatch/1-gap hits elsewhere in the genome. Under this definition, 79.3% of human genome are unique for 35bp reads. 92.4% for 75bp reads. I don't have the number for 150bp reads. I guess it will be around 95%. Empirically, 94-95% of human genome is callable with 100bp paired-end reads.</p>
  
  <p>As to other measurements, the most common one is the fraction of reads that has an exact hit elsewhere. You can use Fabio's method. It gives a good enough estimate once you simulate over 1 million reads. This fraction is around 86% for 35bp reads and 95% for 75bp reads, as I remember. The problem with this approach is this fraction is not very informative to variant calling due to the issues I talked about. Another way is to use RepeatMasker. It is worse. RepeatMasker masks 50% of human genome, but excludes segmental duplications where short reads can't be confidently mapped.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-20T12:54:39.527Z
Answers.Text: "For 150bp reads, each reference position is covered by 150 reads." sounds wrong to me. This is what I think should be in the sentence "For 150bp reads, each read mapped to reference has a 150 positions."
Answers.UserDisplayName: Kamil S Jaron
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-20T21:49:57.537Z
Answers.Text: @KamilSJaron think about 150X coverage.
Answers.UserDisplayName: user172818
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-21T09:53:02.547Z
Answers.Text: Ah, all right, now I get it. From the sentence, I thought that this supposed to be a property of 150bp reads.
Answers.UserDisplayName: Kamil S Jaron
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-19T19:20:48.830Z
Answers.LastActivityDate: 2017-07-19T19:20:48.830Z
Answers.UserDisplayName: user172818
Answers.Body: <p>You might want to check out this paper:</p>
  <ul>
  <li><a href="https://www.ncbi.nlm.nih.gov/pmc/articles/pmid/31104630/" rel="nofollow noreferrer">Ebbert MTW, <em>et al.</em>, Systematic analysis of dark and camouflaged genes reveals disease-relevant genes hiding in plain sight. <em>Genome Biol.</em> 2019 May 20;20(1):97. doi: 10.1186/s13059-019-1707-2. PMID: 31104630; PMCID: PMC6526621.</a></li>
  </ul>
  <p>They have collected a list of &quot;dark&quot; (regions that have few reads aligning to them) and &quot;camouflaged&quot; (regions that have ambiguous alignment) regions of the human genome:</p>
  <blockquote>
  <p>Here, we identify regions with few mappable reads that we call dark by depth, and others that have ambiguous alignment, called camouflaged.</p>
  <p>[. . .]</p>
  <p>Other dark regions arise, not because the sequencing is inherently problematic, but because of bioinformatic challenges. Specifically, many dark regions arise from duplicated genomic regions, where confidently aligning short reads to a unique location is not possible; we term these regions as “camouflaged”. These camouflaged regions are generally either large contiguous tandem repeats (e.g., centromeres, telomeres, and other short tandem repeats), or a larger specific DNA region that has been duplicated (e.g., a gene duplication) either in tandem or in a more distal genome region. In fact, many genes in the human genome were duplicated over evolutionary time and are still transcriptionally and translationally active (e.g., heat-shock proteins) [3–9], while others have been duplicated, but are considered inactive (i.e., pseudogenes). Regardless of whether the duplication is active, however, any genomic region that has been nearly identically duplicated and is large enough to prevent sequencing reads from aligning unambiguously will be “dark”, because the aligner cannot determine which genomic region the read originated from.</p>
  </blockquote>
  <p>The authors have <a href="https://github.com/mebbert/Dark_and_Camouflaged_genes" rel="nofollow noreferrer">made their scripts available</a> on github, along with bed files for the relevant regions. The answer to your question, i.e. a specific percentage, is hard to give since it varies enormously with the genome build and sequencing technology, but this paper should have all the details you want. Have a look at Table 1:</p>
  <p><a href="https://i.sstatic.net/jJN36.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/jJN36.png" alt="Dark and camouflaged regions vary by genome build. We identified dark and camouflaged regions throughout the genome for three different builds, including GRCh37, GRCh38, and GRCh38+alt, across five different sequencing technologies (or read lengths for Illumina). Specifically, we measured dark regions for Illumina based on 100-nucleotide read lengths, Illumina based on 250-nucleotide read lengths, 10x Genomics, PacBio, and Oxford Nanopore Technologies (ONT). Here, the counts for dark and camouflaged regions are combined. We found that the number of dark regions and nucleotides, both within gene bodies (represented as GB in the table) and outside gene bodies, varies dramatically by build and technology. Overall, each technology has its respective strengths. GRCh38 including alternate contigs has &gt; 3x more dark nucleotides than GRCh37, and more than 2x more dark regions. Results presented throughout the manuscript are based on GRCh38 (in gray)" /></a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-02T11:28:56.360Z
Answers.LastActivityDate: 2024-01-02T11:28:56.360Z
Answers.UserDisplayName: terdon
Answers.Body: <p>I would suggest you to run some simulation. You can use <a href="https://github.com/lh3/wgsim" rel="nofollow noreferrer">wgsim</a> to simulate reads that are as much similar to what you want an answer for.
  I don't think there are faster methods.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-19T15:58:14.833Z
Answers.Text: I don’t think this is an appropriate approach to investigate mappability because the sample will be too small. Tools like GEM perform exhaustive calculations.
Answers.UserDisplayName: Konrad Rudolph
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-19T15:02:52.070Z
Answers.LastActivityDate: 2017-07-19T15:02:52.070Z
Answers.UserDisplayName: Fabio Marroni
Body: <p>When doing Illumina 2x150bp sequencing of genomic DNA, and after aligning the reads to GRCh38, what percentage of the non-N fraction of the human genome is MAPQ=0? This is, what part corresponds to regions that can't be uniquely mapped with 2x150bp reads.</p>
  
  <p>And, how many genes are affected by MAPQ=0 regions?</p>
  
  <p>I presume the numbers will dance around depending on fragment sizes, read quality, etc. but I am happy with some starting numbers.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-07-17T12:27:04.117Z
Comments.Text: Are you expecting someone to have these numbers handy or do you want to know how to figure this out? :)
Comments.UserDisplayName: Devon Ryan
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-07-17T12:36:15.013Z
Comments.Text: UCSC used to have downloadable mappability tracks. These don’t seem to exist any more; the best bet now is to [generate them yourself, using GEM](http://wiki.bits.vib.be/index.php/Create_a_mappability_track#Install_and_run_the_GEM_library_tools).
Comments.UserDisplayName: Konrad Rudolph
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-07-18T18:29:03.233Z
Comments.Text: You can start with the repeat masked regions of the genome.  That would reduce the search space if you want to look at 300bp regions.  What is the insert size of your 2x150 library?  That will play a large role in the mappability of the pair.
Comments.UserDisplayName: Bioathlete
Tags: <genome><read-mapping><human><illumina><mapq>
Title: what percentage of the human genome is MAPQ=0?
ThreadText: # what percentage of the human genome is MAPQ=0?
  
  (Asked by: username_0 on 2017-07-17)
  
  When doing Illumina 2x150bp sequencing of genomic DNA, and after aligning the reads to GRCh38, what percentage of the non-N fraction of the human genome is MAPQ=0? This is, what part corresponds to regions that can't be uniquely mapped with 2x150bp reads.
  
  And, how many genes are affected by MAPQ=0 regions?
  
  I presume the numbers will dance around depending on fragment sizes, read quality, etc. but I am happy with some starting numbers.
  
  **Tags:** genome, read-mapping, human, illumina, mapq
  
  > **Comments:**
  > **username_1**: Are you expecting someone to have these numbers handy or do you want to know how to figure this out? :)  
  > **username_2**: UCSC used to have downloadable mappability tracks. These don’t seem to exist any more; the best bet now is to [generate them yourself, using GEM](http://wiki.bits.vib.be/index.php/Create_a_mappability_track#Install_and_run_the_GEM_library_tools).  
  > **username_3**: You can start with the repeat masked regions of the genome.  That would reduce the search space if you want to look at 300bp regions.  What is the insert size of your 2x150 library?  That will play a large role in the mappability of the pair.  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_5 on 2017-07-19. Score: 6
  
  Mapping quality is determined by the repetitiveness of the genome, the sequencing error rate, insert size, the capability of the mapper and the nasty heuristics behind the mapper. MAPQ=0 to one mapper is not necessarily MAPQ=0 to another.
  
  That said, I get what you mean. You want to know the uniqueness/repetitiveness. It is still hard if you want to get a useful answer. For 150bp reads, each reference position is covered by 150 reads. What if 50 of them have no other exact hits elsewhere, but the rest 100 have? Is this position a repeat or not? In addition, what if the 50 are unique only because one mismatch? If there is a variant at that mismatch, the 50 would become repeats or mapped elsewhere.
  
  My preference is to say a position is "unique" under k-long reads if over k/2 reads overlapping the position have no other perfect or 1-mismatch/1-gap hits elsewhere in the genome. Under this definition, 79.3% of human genome are unique for 35bp reads. 92.4% for 75bp reads. I don't have the number for 150bp reads. I guess it will be around 95%. Empirically, 94-95% of human genome is callable with 100bp paired-end reads.
  
  As to other measurements, the most common one is the fraction of reads that has an exact hit elsewhere. You can use username_8's method. It gives a good enough estimate once you simulate over 1 million reads. This fraction is around 86% for 35bp reads and 95% for 75bp reads, as I remember. The problem with this approach is this fraction is not very informative to variant calling due to the issues I talked about. Another way is to use RepeatMasker. It is worse. RepeatMasker masks 50% of human genome, but excludes segmental duplications where short reads can't be confidently mapped.
  
  > **Comments:**
  > **username_6**: "For 150bp reads, each reference position is covered by 150 reads." sounds wrong to me. This is what I think should be in the sentence "For 150bp reads, each read mapped to reference has a 150 positions."  
  > **username_5**: @username_6 think about 150X coverage.  
  > **username_6**: Ah, all right, now I get it. From the sentence, I thought that this supposed to be a property of 150bp reads.  
  
  ---
  
  ### Answer by username_4 on 2017-07-20. Score: 5
  
  As has been said before, mappability to the 'human genome' depends on a number of factors, among these the reference version and type of reads, for which you are interested in GRCh38 and 2x150bp reads. Although I am not aware of numbers accounting for these particular reference and type of Illumina reads, the 1000 genomes project has provided the community with a similar and close estimation that you might be interested in considering regarding your inquiry.
  
  Similar to your question, the 1K project estimated 'the proportion of the human genome that is less accessible to short reads'. In these estimates the human genome is GRCh37 and the types of reads in question are mostly 2x Illumina with a mixture of lengths with the longest being up to 250bp. In these estimates each base in the human genome is considered (and marked) 'less accesible' according to these criteria:
  
  L - depth of coverage is much lower than average
  
  H - depth of coverage is much higher than average
  
  Z - too many reads with zero mapping quality overlap this position
  
  Q - the average mapping quality at the position is too low
  
  Each of these criteria has "standard" and "strict" thresholds for a base to be considered - or not - in each category. You can read more in the link below:
  
  <ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/accessible_genome_masks/README.accessible_genome_mask.20140520>
  
  According to the strict thresholds, the human genome has about 16.8% of "Z" and 3.1% of "Q" bases, respectively. Considering the "Z" and "Q" criteria as a proxy for ~ mapq=0, about 19.9% of the human genome can not be uniquely mapped.
  
  
  ---
  
  ### Answer by username_7 on 2024-01-02. Score: 1
  
  You might want to check out this paper:
  
  * [Ebbert MTW, *et al.*, Systematic analysis of dark and camouflaged genes reveals disease-relevant genes hiding in plain sight. *Genome Biol.* 2019 May 20;20(1):97. doi: 10.1186/s13059-019-1707-2. PMID: 31104630; PMCID: PMC6526621.](https://www.ncbi.nlm.nih.gov/pmc/articles/pmid/31104630/)
  
  They have collected a list of "dark" (regions that have few reads aligning to them) and "camouflaged" (regions that have ambiguous alignment) regions of the human genome:
  
  > Here, we identify regions with few mappable reads that we call dark by depth, and others that have ambiguous alignment, called camouflaged.
  >
  > [. . .]
  >
  > Other dark regions arise, not because the sequencing is inherently problematic, but because of bioinformatic challenges. Specifically, many dark regions arise from duplicated genomic regions, where confidently aligning short reads to a unique location is not possible; we term these regions as “camouflaged”. These camouflaged regions are generally either large contiguous tandem repeats (e.g., centromeres, telomeres, and other short tandem repeats), or a larger specific DNA region that has been duplicated (e.g., a gene duplication) either in tandem or in a more distal genome region. In fact, many genes in the human genome were duplicated over evolutionary time and are still transcriptionally and translationally active (e.g., heat-shock proteins) [3–9], while others have been duplicated, but are considered inactive (i.e., pseudogenes). Regardless of whether the duplication is active, however, any genomic region that has been nearly identically duplicated and is large enough to prevent sequencing reads from aligning unambiguously will be “dark”, because the aligner cannot determine which genomic region the read originated from.
  
  The authors have [made their scripts available](https://github.com/mebbert/Dark_and_Camouflaged_genes) on github, along with bed files for the relevant regions. The answer to your question, i.e. a specific percentage, is hard to give since it varies enormously with the genome build and sequencing technology, but this paper should have all the details you want. Have a look at Table 1:
  
  [![Dark and camouflaged regions vary by genome build. We identified dark and camouflaged regions throughout the genome for three different builds, including GRCh37, GRCh38, and GRCh38+alt, across five different sequencing technologies (or read lengths for Illumina). Specifically, we measured dark regions for Illumina based on 100-nucleotide read lengths, Illumina based on 250-nucleotide read lengths, 10x Genomics, PacBio, and Oxford Nanopore Technologies (ONT). Here, the counts for dark and camouflaged regions are combined. We found that the number of dark regions and nucleotides, both within gene bodies (represented as GB in the table) and outside gene bodies, varies dramatically by build and technology. Overall, each technology has its respective strengths. GRCh38 including alternate contigs has > 3x more dark nucleotides than GRCh37, and more than 2x more dark regions. Results presented throughout the manuscript are based on GRCh38 (in gray)](https://i.sstatic.net/jJN36.png)](https://i.sstatic.net/jJN36.png)
  
  
  ---
  
  ### Answer by username_8 on 2017-07-19. Score: 1
  
  I would suggest you to run some simulation. You can use [wgsim](https://github.com/lh3/wgsim) to simulate reads that are as much similar to what you want an answer for.
  I don't think there are faster methods.
  
  > **Comments:**
  > **username_2**: I don’t think this is an appropriate approach to investigate mappability because the sample will be too small. Tools like GEM perform exhaustive calculations.  
  
  ---
  
  