Answers.Body: <p>The UCSC Genome Browser renders data visually with a one-based, closed index. The underlying data are often stored and presented as files worked with on the command-line, using zero-based, half-open coordinates.</p>
  <p>This difference in indexing can be important when using the website to view data or when using the download tools to grab a dataset for offline calculations, such as files in BED format.</p>
  <p>There is a <a href="https://www.biostars.org/p/84686/" rel="nofollow noreferrer">good overview of indexing</a> on Biostars.</p>
  <p>To solve this in a general way:</p>
  <p>The <a href="http://hgdownload.soe.ucsc.edu/admin/exe/" rel="nofollow noreferrer">UCSC Kent Utilities</a> <code>fetchChromSizes</code> tool can be used to retrieve the sizes of chromosomes for your genome assembly of interest, e.g. <code>hg38</code>.</p>
  <p>Using this assembly, to build a sorted BED file for <a href="https://github.com/bedops/bedops" rel="nofollow noreferrer">set operations</a>:</p>
  <pre><code><span class="math-container">$ fetchChromSizes hg38 | grep -v '_*_' | awk -v FS="\t" -v OFS="\t" '{ print $1, "0", $</span>2 }' | sort-bed - &gt; hg38.bed
  </code></pre>
  <p>Replace <code>hg38</code> with your assembly of interest.</p>
  <p>To then filter this for <code>chrX</code>:</p>
  <pre><code><span class="math-container">$ awk '($</span>1==&quot;chrX&quot;)' hg38.bed &gt; hg38.chrX.bed
  </code></pre>
  <p>The intervals in <code>hg38.bed</code> and <code>hg38.chrX.bed</code> use a zero-based, half-open coordinate system.</p>
  <p>To then get regions of <code>chrX</code> without autosomal regions, you could define a sorted BED file containing those regions, which also use zero-based, half-open coordinates:</p>
  <pre><code>$ echo -e 'chrX\t10000\t2781479\nchrX\t155701382\t156030895' &gt; pseudoautosomal.chrX.bed
  </code></pre>
  <p>I am assuming that the coordinates on Ensembl and NCBI use one-based indexing. Converting to a zero-based index requires subtracting one base from the start positions of these intervals.</p>
  <p>It would be ideal if these groups were to clarify the indexing they use. I am making this assumption based on <a href="https://uswest.ensembl.org/info/genome/genebuild/human_PARS.html" rel="nofollow noreferrer">how Ensembl defines regions</a> on <code>chrY</code>, which start at position <code>1</code> and not <code>0</code>, as one would expect to see with a one-based coordinate system.</p>
  <p>The numbers reported on Ensembl and NCBI are otherwise identical, so I also assume that NCBI's numbers use the same indexing and require the same correction to the start base.</p>
  <p>In any case, to <a href="https://bedops.readthedocs.io/en/latest/content/reference/set-operations/bedops.html#difference-d-difference" rel="nofollow noreferrer">calculate the difference</a> between these regions and <code>chrX</code>:</p>
  <pre><code>$ bedops --difference hg38.chrX.bed pseudoautosomal.chrX.bed &gt; answer.bed
  </code></pre>
  <p>This is what my output looks like:</p>
  <pre><code>$ more answer.bed
  chrX    0   10000
  chrX    2781479 155701382
  chrX    156030895   156040895
  </code></pre>
  <p>These coordinates are in BED format and so use a <a href="https://www.biostars.org/p/84686/" rel="nofollow noreferrer">zero-based, half-open</a> coordinate system, by convention.</p>
  <p>To turn these into one-based, closed coordinates — to query signals over exact regions in the UCSC Genome Browser, for instance — you would add one base to the start positions of each genomic interval.</p>
  <p>(Edit: Fixed PAR#2 start coordinate per comment.)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-17T23:00:18.007Z
Answers.Text: Thank you, great answer! I will look more into that, to understand why our results differ. I could spot one mistake on mine, I missed the 10k bp at the end.
Answers.UserDisplayName: gc5
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-18T01:28:17.220Z
Answers.Text: Another caveat if you work with a mix of data from UCSC and Ensembl is that they pick different naming schemes for chromosomes. For instance, `chrX` is `X` in Ensembl world. If you use set operation tools or `awk`, Python scripts, etc. to process BED files, the name scheme you use should be consistent over files, either one or the other, in order to get correct results. This is easy to fix, but can also be easy to overlook.
Answers.UserDisplayName: Alex Reynolds
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-20T16:38:34.023Z
Answers.Text: Is there a problem with your command: `echo -e 'chrX\t10000\t2781479\nchrX\t155701383\t156030895' > pseudoautosomal.chrX.bed`? Why in the first line the start is `10000` (Start in PAR#1, - 1) and the second line is `155701383` (Start in PAR#2, without -1)? To summarize, why you subtracted 1 from the first line "Start" and not from the second? Thanks! (I am assuming you got the "Start" numbers from https://www.ncbi.nlm.nih.gov/grc/human)
Answers.UserDisplayName: gc5
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-17T21:39:26.910Z
Answers.LastActivityDate: 2021-09-20T17:55:07.337Z
Answers.LastEditDate: 2021-09-20T17:55:07.337Z
Answers.UserDisplayName: Alex Reynolds
Body: <p>I would like to define a bedfile for chromosome X, to target only basepairs outside of pseudo-autosomal regions (and going forward other regions, too).</p>
  <p>First, I wasn't able to find an already prepared bedfile in any of the online repositories (but let me know if it is available somewhere).</p>
  <p>Second, I compiled the bedfile for all basepairs in chromosome X as a starting point (by looking at the latest assembly at <a href="https://genome.ucsc.edu/" rel="nofollow noreferrer">genome browser</a>):</p>
  <pre><code>chrX    0       156040895       # as defined at https://genome.ucsc.edu/
  </code></pre>
  <p>The genome browser is 1-based and the last base is inclusive, while the bedfile is 0-based and last base is not inclusive, therefore I had to subtract 1 from the bed <code>start</code>.</p>
  <p>Afterwards, I wanted to exclude the pseudo-autosomal regions as defined at <a href="https://www.ncbi.nlm.nih.gov/grc/human" rel="nofollow noreferrer">this page</a>. However, I am not sure if basepair &quot;Start&quot; and &quot;Stop&quot; are 0- or 1-based and inclusive/not inclusive. I might use the same convention used on the genome browser, but I am not sure if that's correct in this case. I wasn't able to find this information on their website or online.</p>
  <p>Assuming that the NCBI &quot;Start&quot; and &quot;Stop&quot; field follow the UCSC parameters (1-based and inclusive), the resulting bed (PAR regions excluded) might be the following:</p>
  <pre><code>chrX    0       10000
  chrX    2781479 155701382
  </code></pre>
  <p><strong>Edit</strong>: the section on the centromeres has the &quot;Length&quot; parameter, from which I could infer that &quot;Start&quot; and &quot;Stop&quot; base-pairs are both inclusive. At the same time, the &quot;Stop&quot; base-pair of PAR#2 in chromosome X is base-pair 156,030,895, which is also the last base-pair number if we were considering the UCSC genome browser notation (1-based and inclusive). Therefore, it seems to me clear that the same notation is used, and the proposed bed-file should be correct.</p>
  
Tags: <bed><chromosomes>
Title: Defining a bedfile for chromosome X without pseudo-autosomal regions
ThreadText: # Defining a bedfile for chromosome X without pseudo-autosomal regions
  
  (Asked by: username_0 on 2021-09-17)
  
  I would like to define a bedfile for chromosome X, to target only basepairs outside of pseudo-autosomal regions (and going forward other regions, too).
  
  First, I wasn't able to find an already prepared bedfile in any of the online repositories (but let me know if it is available somewhere).
  
  Second, I compiled the bedfile for all basepairs in chromosome X as a starting point (by looking at the latest assembly at [genome browser](https://genome.ucsc.edu/)):
  
  ```
  chrX    0       156040895       # as defined at https://genome.ucsc.edu/
  
  ```
  
  The genome browser is 1-based and the last base is inclusive, while the bedfile is 0-based and last base is not inclusive, therefore I had to subtract 1 from the bed `start`.
  
  Afterwards, I wanted to exclude the pseudo-autosomal regions as defined at [this page](https://www.ncbi.nlm.nih.gov/grc/human). However, I am not sure if basepair "Start" and "Stop" are 0- or 1-based and inclusive/not inclusive. I might use the same convention used on the genome browser, but I am not sure if that's correct in this case. I wasn't able to find this information on their website or online.
  
  Assuming that the NCBI "Start" and "Stop" field follow the UCSC parameters (1-based and inclusive), the resulting bed (PAR regions excluded) might be the following:
  
  ```
  chrX    0       10000
  chrX    2781479 155701382
  
  ```
  
  **Edit**: the section on the centromeres has the "Length" parameter, from which I could infer that "Start" and "Stop" base-pairs are both inclusive. At the same time, the "Stop" base-pair of PAR#2 in chromosome X is base-pair 156,030,895, which is also the last base-pair number if we were considering the UCSC genome browser notation (1-based and inclusive). Therefore, it seems to me clear that the same notation is used, and the proposed bed-file should be correct.
  
  **Tags:** bed, chromosomes
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-09-17. Score: 2 | Accepted answer
  
  The UCSC Genome Browser renders data visually with a one-based, closed index. The underlying data are often stored and presented as files worked with on the command-line, using zero-based, half-open coordinates.
  
  This difference in indexing can be important when using the website to view data or when using the download tools to grab a dataset for offline calculations, such as files in BED format.
  
  There is a [good overview of indexing](https://www.biostars.org/p/84686/) on Biostars.
  
  To solve this in a general way:
  
  The [UCSC Kent Utilities](http://hgdownload.soe.ucsc.edu/admin/exe/) `fetchChromSizes` tool can be used to retrieve the sizes of chromosomes for your genome assembly of interest, e.g. `hg38`.
  
  Using this assembly, to build a sorted BED file for [set operations](https://github.com/bedops/bedops):
  
  ```
  $ fetchChromSizes hg38 | grep -v '_*_' | awk -v FS="\t" -v OFS="\t" '{ print $1, "0", $2 }' | sort-bed - > hg38.bed
  
  ```
  
  Replace `hg38` with your assembly of interest.
  
  To then filter this for `chrX`:
  
  ```
  $ awk '($1=="chrX")' hg38.bed > hg38.chrX.bed
  
  ```
  
  The intervals in `hg38.bed` and `hg38.chrX.bed` use a zero-based, half-open coordinate system.
  
  To then get regions of `chrX` without autosomal regions, you could define a sorted BED file containing those regions, which also use zero-based, half-open coordinates:
  
  ```
  $ echo -e 'chrX\t10000\t2781479\nchrX\t155701382\t156030895' > pseudoautosomal.chrX.bed
  
  ```
  
  I am assuming that the coordinates on Ensembl and NCBI use one-based indexing. Converting to a zero-based index requires subtracting one base from the start positions of these intervals.
  
  It would be ideal if these groups were to clarify the indexing they use. I am making this assumption based on [how Ensembl defines regions](https://uswest.ensembl.org/info/genome/genebuild/human_PARS.html) on `chrY`, which start at position `1` and not `0`, as one would expect to see with a one-based coordinate system.
  
  The numbers reported on Ensembl and NCBI are otherwise identical, so I also assume that NCBI's numbers use the same indexing and require the same correction to the start base.
  
  In any case, to [calculate the difference](https://bedops.readthedocs.io/en/latest/content/reference/set-operations/bedops.html#difference-d-difference) between these regions and `chrX`:
  
  ```
  $ bedops --difference hg38.chrX.bed pseudoautosomal.chrX.bed > answer.bed
  
  ```
  
  This is what my output looks like:
  
  ```
  $ more answer.bed
  chrX    0   10000
  chrX    2781479 155701382
  chrX    156030895   156040895
  
  ```
  
  These coordinates are in BED format and so use a [zero-based, half-open](https://www.biostars.org/p/84686/) coordinate system, by convention.
  
  To turn these into one-based, closed coordinates — to query signals over exact regions in the UCSC Genome Browser, for instance — you would add one base to the start positions of each genomic interval.
  
  (Edit: Fixed PAR#2 start coordinate per comment.)
  
  > **Comments:**
  > **username_0**: Thank you, great answer! I will look more into that, to understand why our results differ. I could spot one mistake on mine, I missed the 10k bp at the end.  
  > **username_1**: Another caveat if you work with a mix of data from UCSC and Ensembl is that they pick different naming schemes for chromosomes. For instance, `chrX` is `X` in Ensembl world. If you use set operation tools or `awk`, Python scripts, etc. to process BED files, the name scheme you use should be consistent over files, either one or the other, in order to get correct results. This is easy to fix, but can also be easy to overlook.  
  > **username_0**: Is there a problem with your command: `echo -e 'chrX\t10000\t2781479\nchrX\t155701383\t156030895' > pseudoautosomal.chrX.bed`? Why in the first line the start is `10000` (Start in PAR#1, - 1) and the second line is `155701383` (Start in PAR#2, without -1)? To summarize, why you subtracted 1 from the first line "Start" and not from the second? Thanks! (I am assuming you got the "Start" numbers from https://www.ncbi.nlm.nih.gov/grc/human)  
  
  ---
  
  