Answers.Body: <p>The "right" solution would be realignment, but that's expensive and most of us would not go that route. My preferred solution would be to convert the bed file, as opposed to the bam. Here's why:</p>
  
  <p>1) Reheadering the bam means that you may have reads aligned to contigs without a corresponding entry in UCSC (see <a href="https://github.com/dpryan79/ChromosomeMappings/blob/master/GRCh37_NCBI2UCSC.txt" rel="nofollow noreferrer">Devon's list for the mappings</a>). This is a problem because:</p>
  
  <ul>
  <li>Some of those reads would likely have been mapped elsewhere if a reference without those contigs was used. </li>
  <li>I'm not even sure what happens to those reads after reheadering - I guess they would need to be marked as unmapped? Lots of potential for screwiness there.</li>
  </ul>
  
  <p>2) It seems cleaner to convert the bed file from UCSC->NCBI, where you are guaranteed that every entry has a "home". Then, after you pull your info from the bam, you can always convert chromosome names back if you need to.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-08T11:03:14.860Z
Answers.Text: Several tools will throw a wobbly if you pass reads on `*` that are not marked as unmapped. `pysam` is is one. I know this from bitter (bug fixing ) experience.
Answers.UserDisplayName: Ian Sudbery
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-17T19:02:03.413Z
Answers.Text: Alignments that end up losing their chromosome end up unmapped, though they don't get the SAM unmapped flag (rather, they end up represented as being on the `*` chromosome). But yeah, `samtools reheader` should be used with extreme caution.
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-17T18:56:13.670Z
Answers.LastActivityDate: 2017-05-17T18:56:13.670Z
Answers.UserDisplayName: chrisamiller
Answers.Body: <p>Just to point out that if you want to follow @Devon Ryan's answer for a different organism/assembly, that is not in his very useful linked resource, you can download NCBI to UCSC contig to chromosome number mappings from <a href="https://www.ncbi.nlm.nih.gov/assembly" rel="nofollow noreferrer">https://www.ncbi.nlm.nih.gov/assembly</a>.</p>
  
  <p>To the site and search for your assembly name. At the bottom of the page is a box called "Global assembly definition" containing a link titled "Download full sequence report".</p>
  
  <p>The downloaded file contains a table with:</p>
  
  <ul>
  <li>Chromosome numbers in "Sequence-Name"/"Assigned-molecule"  </li>
  <li>NCBI names in Refseq-Accn  </li>
  <li>UCSC contig name in "UCSC-style-name"</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-08T11:02:02.630Z
Answers.LastActivityDate: 2017-08-08T11:02:02.630Z
Answers.UserDisplayName: Ian Sudbery
Answers.Body: <p>You're the second person I have ever seen using NCBI "chromosome names" (they're more like supercontig IDs). Normally I would point you to <a href="https://github.com/dpryan79/ChromosomeMappings" rel="noreferrer">a resource providing mappings between chromosome names</a>, but since no one has added NCBI names (yet, maybe I'll add them now) you're currently out of luck there.</p>
  
  <p>Anyway, the quickest way to do what you want is to <code>samtools view -H foo.bam &gt; header</code> to get the BAM header and then change each NCBI "chromosome name" to its corresponding UCSC chromosome name. DO NOT REORDER THE LINES! You can then use <code>samtools reheader</code> and be done.</p>
  
  <p>Why, you might ask, would this work? The answer is that chromosome/contig names in BAM files aren't stored in each alignment. Rather, the names are stored in a list in the header and each alignment just contains the integer index into that list (read group IDs are similar, for what it's worth). This also leads to the warning above against reordering entries, since that's a VERY convenient way to start swapping alignments between chromosomes.</p>
  
  <p>As an aside, you'd be well served switching to Gencode or Ensembl chromosome names, they're rather more coherent than the <code>something_random</code> mess that's present in hg19 from UCSC.</p>
  
  <p><strong>Update</strong>: Because I'm nice, <a href="https://github.com/dpryan79/ChromosomeMappings/blob/master/GRCh37_NCBI2UCSC.txt" rel="noreferrer">here</a> is the conversion between NCBI and UCSC. Note that if you have any alignments to patches that there is simply no UCSC equivalent. One of the many reasons not to use UCSC (avoid their annotations too).</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-18T10:52:43.450Z
Answers.Text: How well does this work? Have you run any benchmarks? I ask because I tried to convert various bed files between hg and GRC genomes and the three tools I used all gave very different results. This sort of mapping really should be a simple thing but it appears not to be that straightforward at all.
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-18T11:10:54.857Z
Answers.Text: For cases where it's just a change of name (most cases) there's nothing to benchmark. For cases where you additionally have position changes then you'd need a different resource (namely, liftOver or crossmap).
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-18T11:19:55.757Z
Answers.Text: Yeah, it's tolls like liftOver and crossmap I've used and found issues with. I was expecting this to be a solved problem but each of the three tools I used gave different results, unfortunately. Which makes me leery of using the results.
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-18T11:20:59.213Z
Answers.Text: The results are deterministic, they should be the same regardless of tool provided you use the same settings.
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-18T11:26:56.163Z
Answers.Text: You'd think so, yes. But they're not. I'd be happy to chat ([chat]) about this (this is one of things I need to sort out at work) if you like, and I'd be very pleased to learn I was just doing it wrong, but my preliminary tests and various threads I've read online (see [here](https://www.biostars.org/p/14187/#91490) for example) suggest it isn't as simple as you'd think.
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-18T11:40:15.927Z
Answers.Text: I suspect this will come down to deficiencies in the chain files, but I'd like to hear about that.
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-16T21:04:42.503Z
Answers.LastActivityDate: 2017-05-16T21:46:53.327Z
Answers.LastEditDate: 2017-05-16T21:46:53.327Z
Answers.UserDisplayName: Devon Ryan
Body: <p>I have a set of BAM files that are aligned using the NCBI GRCh37 human genome reference (with the chromosome names as NC_000001.10) but I want to analyze it using a BED file that has the UCSC hg19 chromosome names (e.g. chr1). I want to use bedtools to pull out all the on-target and off-target reads.</p>
  
  <ol>
  <li>Are NCBI and UCSC directly comparable? Or do I need to re-align the BAM/lift-over the BED to the UCSC reference?</li>
  <li>Should I convert the BED file or the BAM file? Everyone here uses the UCSC chromosome names/positions so I'll need to convert the eventual files to UCSC anyway.</li>
  </ol>
  
Tags: <sam><file-formats><bed><format-conversion><reference-genome>
Title: Convert a BAM file from one reference to another?
ThreadText: # Convert a BAM file from one reference to another?
  
  (Asked by: username_0 on 2017-05-16)
  
  I have a set of BAM files that are aligned using the NCBI GRCh37 human genome reference (with the chromosome names as NC\_000001.10) but I want to analyze it using a BED file that has the UCSC hg19 chromosome names (e.g. chr1). I want to use bedtools to pull out all the on-target and off-target reads.
  
  1. Are NCBI and UCSC directly comparable? Or do I need to re-align the BAM/lift-over the BED to the UCSC reference?
  2. Should I convert the BED file or the BAM file? Everyone here uses the UCSC chromosome names/positions so I'll need to convert the eventual files to UCSC anyway.
  
  **Tags:** sam, file-formats, bed, format-conversion, reference-genome
  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_3 on 2017-05-16. Score: 24 | Accepted answer
  
  You're the second person I have ever seen using NCBI "chromosome names" (they're more like supercontig IDs). Normally I would point you to [a resource providing mappings between chromosome names](https://github.com/dpryan79/ChromosomeMappings), but since no one has added NCBI names (yet, maybe I'll add them now) you're currently out of luck there.
  
  Anyway, the quickest way to do what you want is to `samtools view -H foo.bam > header` to get the BAM header and then change each NCBI "chromosome name" to its corresponding UCSC chromosome name. DO NOT REORDER THE LINES! You can then use `samtools reheader` and be done.
  
  Why, you might ask, would this work? The answer is that chromosome/contig names in BAM files aren't stored in each alignment. Rather, the names are stored in a list in the header and each alignment just contains the integer index into that list (read group IDs are similar, for what it's worth). This also leads to the warning above against reordering entries, since that's a VERY convenient way to start swapping alignments between chromosomes.
  
  As an aside, you'd be well served switching to Gencode or Ensembl chromosome names, they're rather more coherent than the `something_random` mess that's present in hg19 from UCSC.
  
  **Update**: Because I'm nice, [here](https://github.com/dpryan79/ChromosomeMappings/blob/master/GRCh37_NCBI2UCSC.txt) is the conversion between NCBI and UCSC. Note that if you have any alignments to patches that there is simply no UCSC equivalent. One of the many reasons not to use UCSC (avoid their annotations too).
  
  > **Comments:**
  > **username_4**: How well does this work? Have you run any benchmarks? I ask because I tried to convert various bed files between hg and GRC genomes and the three tools I used all gave very different results. This sort of mapping really should be a simple thing but it appears not to be that straightforward at all.  
  > **username_3**: For cases where it's just a change of name (most cases) there's nothing to benchmark. For cases where you additionally have position changes then you'd need a different resource (namely, liftOver or crossmap).  
  > **username_4**: Yeah, it's tolls like liftOver and crossmap I've used and found issues with. I was expecting this to be a solved problem but each of the three tools I used gave different results, unfortunately. Which makes me leery of using the results.  
  > **username_3**: The results are deterministic, they should be the same regardless of tool provided you use the same settings.  
  > **username_4**: You'd think so, yes. But they're not. I'd be happy to chat ([chat]) about this (this is one of things I need to sort out at work) if you like, and I'd be very pleased to learn I was just doing it wrong, but my preliminary tests and various threads I've read online (see [here](https://www.biostars.org/p/14187/#91490) for example) suggest it isn't as simple as you'd think.  
  > **username_3**: I suspect this will come down to deficiencies in the chain files, but I'd like to hear about that.  
  
  ---
  
  ### Answer by username_1 on 2017-05-17. Score: 4
  
  The "right" solution would be realignment, but that's expensive and most of us would not go that route. My preferred solution would be to convert the bed file, as opposed to the bam. Here's why:
  
  1) Reheadering the bam means that you may have reads aligned to contigs without a corresponding entry in UCSC (see [username_3's list for the mappings](https://github.com/dpryan79/ChromosomeMappings/blob/master/GRCh37_NCBI2UCSC.txt)). This is a problem because:
  
  * Some of those reads would likely have been mapped elsewhere if a reference without those contigs was used.
  * I'm not even sure what happens to those reads after reheadering - I guess they would need to be marked as unmapped? Lots of potential for screwiness there.
  
  2) It seems cleaner to convert the bed file from UCSC->NCBI, where you are guaranteed that every entry has a "home". Then, after you pull your info from the bam, you can always convert chromosome names back if you need to.
  
  > **Comments:**
  > **username_3**: Alignments that end up losing their chromosome end up unmapped, though they don't get the SAM unmapped flag (rather, they end up represented as being on the `*` chromosome). But yeah, `samtools reheader` should be used with extreme caution.  
  > **username_2**: Several tools will throw a wobbly if you pass reads on `*` that are not marked as unmapped. `pysam` is is one. I know this from bitter (bug fixing ) experience.  
  
  ---
  
  ### Answer by username_2 on 2017-08-08. Score: 0
  
  Just to point out that if you want to follow @username_3's answer for a different organism/assembly, that is not in his very useful linked resource, you can download NCBI to UCSC contig to chromosome number mappings from <https://www.ncbi.nlm.nih.gov/assembly>.
  
  To the site and search for your assembly name. At the bottom of the page is a box called "Global assembly definition" containing a link titled "Download full sequence report".
  
  The downloaded file contains a table with:
  
  * Chromosome numbers in "Sequence-Name"/"Assigned-molecule"
  * NCBI names in Refseq-Accn
  * UCSC contig name in "UCSC-style-name"
  
  
  ---
  
  