Answers.Body: <p>I managed to solve this with a one-liner:</p>
  
  <pre><code>grep -v 'Sample ID' CosmicFusionExport.tsv | cut -f 12 | grep -v -e '^$' | perl -lne '/^(\S+?)\{\S+\d+}\S+_(\S+?)\{/; print "$1\t$2"' | sort -u &gt; CosmicFusionList.tsv 
  </code></pre>
  
  <p>The various parts of the piped command are as follows:</p>
  
  <ol>
  <li>Remove header:<code>grep -v 'Sample ID' CosmicFusionExport.tsv |</code></li>
  <li>Extract 12th field (Translocation Name):<code>cut -f 12 |</code></li>
  <li>Remove blank entries:<code>grep -v -e '^$' |</code></li>
  <li>Extract gene symbols from 12th column:<code>perl -lne '/^(\S+?)\{\S+\d+}\S+_(\S+?)\{/; print "$1\t$2"' |</code></li>
  <li>Remove duplicate lines:<code>sort -u &gt; CosmicFusionList.tsv</code></li>
  <li>Optionally the final two stages could be:<code>sort -u | gzip -c &gt; CosmicFusionList.tsv.gz</code> to make use of file compression (for what it's worth on a 3.9 kB file)</li>
  </ol>
  
  <p>Since the authors of arriba have neglected to document this necessary data munging step, I've posted it here in Q and A format in the hope that it will be useful to others, and other solutions can be proposed.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-08-02T14:05:20.477Z
Answers.LastActivityDate: 2018-08-02T14:05:20.477Z
Answers.UserDisplayName: Matt Bashton
Body: <p>The RNA-Seq fusion detection program <a href="https://github.com/suhrig/arriba" rel="nofollow noreferrer">Arriba</a> features the ability to white-list known gene fusions, specifically the authors recommend using <a href="https://cancer.sanger.ac.uk/cosmic/download?genome=37" rel="nofollow noreferrer">COSMIC</a> as a source of these <a href="http://arriba.readthedocs.io/en/latest/input-files/#known-fusions" rel="nofollow noreferrer">here</a>. Specifically they state:</p>
  
  <blockquote>
    <p>A comprehensive list of known fusions can be obtained from CancerGeneCensus in the section titled "Complete Fusion Export". <em>Depending on the gene annotation that is used to run arriba, some gene names need to be adjusted</em>. </p>
  </blockquote>
  
  <p>Emphasis is mine to highlight the issue here!</p>
  
  <p>The state the format of the input file they require to be:</p>
  
  <blockquote>
    <p>The file has two columns separated by a tab. Each line lists a pair of
    genes. The order of the genes is irrelevant. arriba searches for both
    genes as the 5' end and the 3' end of a fusion. Lines starting with a
    hash (#) are treated as comments. Optionally, the file can be
    gzip-compressed.</p>
  </blockquote>
  
  <p>However the format of the download from COSMIC is somewhat different:</p>
  
  <pre><code>Sample ID       Sample name     Primary site    Site subtype 1  Site subtype 2  Site subtype 3  Primary histology       Histology subtype 1     Histology subtype 2     Histology subtype 3     Fusion ID       Translocation Name      Fusion Type     PUBMED_PMID     ID_STUDY
  WWWWWWW WWWWWWW thyroid NS      NS      NS      carcinoma       papillary_carcinoma     follicular_variant      NS                              16784981
  WWWWWWW WWWWWWW thyroid NS      NS      NS      carcinoma       papillary_carcinoma     follicular_variant      NS      1271    CCDC6{ENST00000263102}:r.1_535_RET{ENST00000355710}:r.2369_5659 Observed mRNA   16784981
  WWWWWWW WWWWWWW thyroid NS      NS      NS      carcinoma       papillary_carcinoma     follicular_variant      NS      1272    CCDC6{ENST00000263102}:r.1_535_RET{ENST00000355710}:r.2369_5659 Inferred Breakpoint     16784981
  XXXXXXX XXXXXXX thyroid NS      NS      NS      carcinoma       papillary_carcinoma     follicular_variant      NS                              16784981
  YYYYYYY YYYYYYY thyroid NS      NS      NS      carcinoma       papillary_carcinoma     follicular_variant      NS                              16728576
  YYYYYYY YYYYYYY thyroid NS      NS      NS      carcinoma       papillary_carcinoma     follicular_variant      NS      1216    PAX8{NM_003466.2}:r.?_PPARG{NM_015869.2}:r.?    Inferred Breakpoint     16728576
  </code></pre>
  
  <p><em>Note: numeric samples IDs have been deliberately replaced with letters here.</em></p>
  
  <p>Using this file, as is, will cause arriba to spurt out many warnings that the gene names in the first two columns are not recognised:</p>
  
  <p><code>WARNING: unknown gene in known fusions list: XXXXXXX</code></p>
  
  <p>From the looks of the rest of my arriba output the currency of gene IDs its using would appear to be gene symbols taken from the <code>gencode.gft</code> I'm supplying it.</p>
  
  <p>I'm thus assuming the format arriba requires for a known gene list to be:</p>
  
  <pre><code>SYMB1   SYMB2
  SYMB3   SYMB4
  </code></pre>
  
  <p>Where the record separator is a tab character <code>\t</code></p>
  
  <p>My question is how can I reprocess the COSMIC "Complete Fusion Export" <a href="https://cancer.sanger.ac.uk/cosmic/download?genome=37" rel="nofollow noreferrer"><code>.tsv</code> file</a> to the format required for arriba to act as a white-list?</p>
  
Tags: <rna-seq><fusions>
Title: Generating list of known fusions for Arriba from COSMIC
ThreadText: # Generating list of known fusions for Arriba from COSMIC
  
  (Asked by: username_0 on 2018-08-02)
  
  The RNA-Seq fusion detection program [Arriba](https://github.com/suhrig/arriba) features the ability to white-list known gene fusions, specifically the authors recommend using [COSMIC](https://cancer.sanger.ac.uk/cosmic/download?genome=37) as a source of these [here](http://arriba.readthedocs.io/en/latest/input-files/#known-fusions). Specifically they state:
  
  > A comprehensive list of known fusions can be obtained from CancerGeneCensus in the section titled "Complete Fusion Export". *Depending on the gene annotation that is used to run arriba, some gene names need to be adjusted*.
  
  Emphasis is mine to highlight the issue here!
  
  The state the format of the input file they require to be:
  
  > The file has two columns separated by a tab. Each line lists a pair of
  > genes. The order of the genes is irrelevant. arriba searches for both
  > genes as the 5' end and the 3' end of a fusion. Lines starting with a
  > hash (#) are treated as comments. Optionally, the file can be
  > gzip-compressed.
  
  However the format of the download from COSMIC is somewhat different:
  
  ```
  Sample ID       Sample name     Primary site    Site subtype 1  Site subtype 2  Site subtype 3  Primary histology       Histology subtype 1     Histology subtype 2     Histology subtype 3     Fusion ID       Translocation Name      Fusion Type     PUBMED_PMID     ID_STUDY
  WWWWWWW WWWWWWW thyroid NS      NS      NS      carcinoma       papillary_carcinoma     follicular_variant      NS                              16784981
  WWWWWWW WWWWWWW thyroid NS      NS      NS      carcinoma       papillary_carcinoma     follicular_variant      NS      1271    CCDC6{ENST00000263102}:r.1_535_RET{ENST00000355710}:r.2369_5659 Observed mRNA   16784981
  WWWWWWW WWWWWWW thyroid NS      NS      NS      carcinoma       papillary_carcinoma     follicular_variant      NS      1272    CCDC6{ENST00000263102}:r.1_535_RET{ENST00000355710}:r.2369_5659 Inferred Breakpoint     16784981
  XXXXXXX XXXXXXX thyroid NS      NS      NS      carcinoma       papillary_carcinoma     follicular_variant      NS                              16784981
  YYYYYYY YYYYYYY thyroid NS      NS      NS      carcinoma       papillary_carcinoma     follicular_variant      NS                              16728576
  YYYYYYY YYYYYYY thyroid NS      NS      NS      carcinoma       papillary_carcinoma     follicular_variant      NS      1216    PAX8{NM_003466.2}:r.?_PPARG{NM_015869.2}:r.?    Inferred Breakpoint     16728576
  
  ```
  
  *Note: numeric samples IDs have been deliberately replaced with letters here.*
  
  Using this file, as is, will cause arriba to spurt out many warnings that the gene names in the first two columns are not recognised:
  
  `WARNING: unknown gene in known fusions list: XXXXXXX`
  
  From the looks of the rest of my arriba output the currency of gene IDs its using would appear to be gene symbols taken from the `gencode.gft` I'm supplying it.
  
  I'm thus assuming the format arriba requires for a known gene list to be:
  
  ```
  SYMB1   SYMB2
  SYMB3   SYMB4
  
  ```
  
  Where the record separator is a tab character `\t`
  
  My question is how can I reprocess the COSMIC "Complete Fusion Export" [`.tsv` file](https://cancer.sanger.ac.uk/cosmic/download?genome=37) to the format required for arriba to act as a white-list?
  
  **Tags:** rna-seq, fusions
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2018-08-02. Score: 1 | Accepted answer
  
  I managed to solve this with a one-liner:
  
  ```
  grep -v 'Sample ID' CosmicFusionExport.tsv | cut -f 12 | grep -v -e '^$' | perl -lne '/^(\S+?)\{\S+\d+}\S+_(\S+?)\{/; print "$1\t$2"' | sort -u > CosmicFusionList.tsv 
  
  ```
  
  The various parts of the piped command are as follows:
  
  1. Remove header:`grep -v 'Sample ID' CosmicFusionExport.tsv |`
  2. Extract 12th field (Translocation Name):`cut -f 12 |`
  3. Remove blank entries:`grep -v -e '^$' |`
  4. Extract gene symbols from 12th column:`perl -lne '/^(\S+?)\{\S+\d+}\S+_(\S+?)\{/; print "$1\t$2"' |`
  5. Remove duplicate lines:`sort -u > CosmicFusionList.tsv`
  6. Optionally the final two stages could be:`sort -u | gzip -c > CosmicFusionList.tsv.gz` to make use of file compression (for what it's worth on a 3.9 kB file)
  
  Since the authors of arriba have neglected to document this necessary data munging step, I've posted it here in Q and A format in the hope that it will be useful to others, and other solutions can be proposed.
  
  
  ---
  
  