Answers.Body: <p>Just use minimap2 in split alignment mode to realign the reads.</p>
  
  <p>If that is not an option, then you could try using <a href="https://pysam.readthedocs.io/" rel="nofollow noreferrer">pysam</a> to modify the CIGAR strings. I do not recommend this, as there are many opportunities for subtle bugs because the SAM spec is complex. You would need to:</p>
  
  <ol>
  <li>Sort the BAM on read ID so that you can efficiently retrieve reads that you want to merge</li>
  <li>Iterate through the BAM with pysam while loading all reads of the same read ID</li>
  <li>For each set of reads with the same read ID:
  
  <ol start="4">
  <li>Order the reads of the same read ID by the number of hard-clipped bases at the start of the CIGAR string.</li>
  <li>Build a new CIGAR string from the ordered CIGAR strings</li>
  <li>Merge the meta tags from the ordered reads</li>
  <li>Write out a single, merged read</li>
  </ol></li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-18T10:28:05.063Z
Answers.Text: I did that at the end. At the time, I just wanted to reformat the non-spliced alignments from LAST (for example) into spliced alignments. Why? Benchmark the most tools possible. But it's ok, I gave up at the end and just used tools that already gave that spliced alignment directly.
Answers.UserDisplayName: aechchiki
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-18T10:53:53.457Z
Answers.Text: That's great. However, the point of SE is to also help others that might have similar issues in the future by providing answers to questions so that users can upvote good answers. If you feel that my answer is a good one to the question you posed, then please consider up voting and accepting it.
Answers.UserDisplayName: winni2k
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-18T10:55:10.860Z
Answers.Text: If you feel that this answer does not answer the question you really had, then please consider rewriting your question for clarity.
Answers.UserDisplayName: winni2k
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-18T08:35:09.460Z
Answers.LastActivityDate: 2018-11-18T08:35:09.460Z
Answers.UserDisplayName: winni2k
Body: <p>I mapped RNA reads to reference genome, using <a href="http://last.cbrc.jp/" rel="noreferrer">LAST</a> in <a href="http://last.cbrc.jp/doc/last-split.html" rel="noreferrer">split mode</a>, and converted the MAF alignment to SAM with <a href="http://last.cbrc.jp/doc/maf-convert.html" rel="noreferrer">maf-convert</a>. </p>
  
  <p>My problem is that the transcripts are not reported in a spliced manner, meaning that a <code>transcript_ID</code> is reported several times in the alignment SAM file with an identical bitwise flag in <code>$2</code>. From what I understand, this is due to the fact that only the exons are mapped (one exon per row) and reported as local alignments, as the software cannot deal with splicing patterns combination exon-intron (yet), which behaviour is clear from the CIGAR string. </p>
  
  <p>For a more concrete and visual example let's consider the mapping of transcript <code>FBtr0344900</code> to the reference genome as given by LAST:</p>
  
  <pre><code>$ cat last_aln.sam | grep FBtr0344900
  FBtr0344900 0   4   42774   100 384=144H    *   0   0   TGCGACATTGTTCTACGATGACTACAAAAAATGACCAATAACTTCTATAAACCAATACGATATGTCAGGAGTTTCGGTCCCATACGAAGTCGCCGACTTAAGTATTTTATttttattttgatATGTGTTTGCTATTTTACCTTGTCGAATGCTTCCACACGCTATGAGAATACCATCGTGAGCGTAGCTTACTACTAGAATTTTGTTGAAGTTATTGACAAGCGATGTCTCAATATCTTCCGGACAGCCTCCAGCGTGACATTGCGGGGAATCATGTAACGGCCCAGTAACAGCCTCGGCCAGCACTCGAAGGTTTTCGTTAAGTTTAAGTATTTTATTTGTAGCACCCGCAAACAAAACATTGTGCATAAAGTCGAAGCTCAT    *   NM:i:0  AS:i:2304
  FBtr0344900 0   4   43231   100 384H144=    *   0   0   CTGGAAGCTGTTGATTGAACTGGTATTGATGGCAAGTTAAACTGGGCGACTATGTCATTTAAGGGAGATAACGCCTGAGCCGGCAGTTCTTCAATGCAGTTAACGCAATAATGCTGAGAACCGAGTATGATAATAATACACAGT    *   NM:i:0  AS:i:864
  </code></pre>
  
  <p>And here is the mapping of the same transcript <code>FBtr0344900</code> as given by <a href="https://github.com/alexdobin/STAR" rel="noreferrer">STAR</a> - software which reports the alignment right the way I need it:</p>
  
  <pre><code>cat star_aln.sam | grep FBtr0344900
  FBtr0344900    0    4    42774    255    384M73N144M    *    0    0TGCGACATTGTTCTACGATGACTACAAAAAATGACCAATAACTTCTATAAACCAATACGATATGTCAGGAGTTTCGGTCCCATACGAAGTCGCCGACTTAAGTATTTTATTTTTATTTTGATATGTGTTTGCTATTTTACCTTGTCGAATGCTTCCACACGCTATGAGAATACCATCGTGAGCGTAGCTTACTACTAGAATTTTGTTGAAGTTATTGACAAGCGATGTCTCAATATCTTCCGGACAGCCTCCAGCGTGACATTGCGGGGAATCATGTAACGGCCCAGTAACAGCCTCGGCCAGCACTCGAAGGTTTTCGTTAAGTTTAAGTATTTTATTTGTAGCACCCGCAAACAAAACATTGTGCATAAAGTCGAAGCTCATCTGGAAGCTGTTGATTGAACTGGTATTGATGGCAAGTTAAACTGGGCGACTATGTCATTTAAGGGAGATAACGCCTGAGCCGGCAGTTCTTCAATGCAGTTAACGCAATAATGCTGAGAACCGAGTATGATAATAATACACAGT    *    NH:i:1    HI:i:1    NM:i:0    MD:Z:528
  </code></pre>
  
  <p>From <a href="https://groups.google.com/forum/#!topic/last-align/xpseSgE7Avs" rel="noreferrer">discussion</a> with the author, it seems that I cannot get what I need straight from the current LAST version, and is not a technical problem. So, I have to modify the output myself. The aim would be to get at least a CIGAR line representing a complete transcript. </p>
  
  <p>My question is, do you know any software to do this? What I would need is a file containing one line per uniquely mapped transcripts, featuring three fields: <code>transcript_ID</code>, <code>start position</code> and <code>CIGAR string</code>.</p>
  
  <p>From my side, I proceeded like this : </p>
  
  <p>1) extract the fields that I need for the  interested in from the SAM file: </p>
  
  <pre><code>$ cut -f1,4,6
  FBtr0344900  42774  384=144H
  FBtr0344900  43231  384H144=
  </code></pre>
  
  <p>2) split the CIGAR line to remove items I am not interested in - I simplify the command here, just assuming I only have perfect matches (which I am interested in) and hard-clipping (which I am not interested in):</p>
  
  <pre><code>$ cut -f3 | sed 's/H/_ /g'| sed 's/=/= /g' | sed 's/\w*_\s*//' | sed 's/=//g'
  384
  144
  </code></pre>
  
  <p>3) paste the modified CIGAR in the original file, with <code>paste</code>, resulting in: </p>
  
  <pre><code>$ paste (1) (2) | cut -f1,2,4
  FBtr0344900  42774  384
  FBtr0344900  43231  144
  </code></pre>
  
  <p>4) merge rows which start ba the same <code>transcript_id</code>: </p>
  
  <pre><code>$ awk -F'\t' -v OFS='\t' '{ x=$1 ; $1="" ; a[x]=a[x]$0 } END { for(x in a) print x,a[x] }' | 
  FBtr0344900    42774    384    43231   144 
  </code></pre>
  
  <p>5) compute the new cigar, calculating the intron length as the arithmetic formula <code>intron_length= (next_exon_start_coordinate - exon_length - previous_exon_start_coordinate)</code>, in this simple case above: <code>intron_length= 43231-384-42774</code> </p>
  
  <pre><code>$ awk '{ printf("%s", $1) }; { for (i=4; i&lt;=NF; i+=2) { printf("%s%d%s%d", OFS, $(i-1), OFS, $i - $(i-1) - $(i-2)) } }; { printf("%s%d%s%s", OFS, $NF, OFS, RS) }'
  FBtr0344900 384 73 144
  </code></pre>
  
  <p>6) ideally, with some method that I will figure out, I will modify the CIGAR string (adding alternate M,N at the end of each field except first one), this is how the final file should look like: </p>
  
  <pre><code>FBtr0344900 42774  384M73N144M
  </code></pre>
  
  <p>The issue of my basic approach are that: </p>
  
  <ol>
  <li>I am not sure how to account for the 1-based SAM : should I add <code>+1</code> at every <code>exon_start_coordinate</code>? Does not look like, as the STAR output has the exact same cigar string that I calculated on the STAR output. </li>
  <li>BIG issue: This will only work for the reads mapped in forward strand: how to make it feasible with reads mapped in the reverse strand? If I keep my current approach, I will have negative intron sizes...</li>
  </ol>
  
  <p>Any suggestion is welcome!</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-08-30T13:28:55.340Z
Comments.Text: I strongly encourage you to not do this with normal unix tools and to instead code something in python (or whatever language you prefer).
Comments.UserDisplayName: Devon Ryan
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-08-30T16:00:06.650Z
Comments.Text: I would use an aligner that outputs proper SAM, such as star, gmap, spaln or minimap2.
Comments.UserDisplayName: user172818
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-08-30T16:31:33.903Z
Comments.Text: yes, I already run the same alignment with GMAP, STAR. we have suspect that LAST might work quite well, but as for now the output is not directly helpful. I will try minimap2 (nice to know you can use it for RNA also)
Comments.UserDisplayName: aechchiki
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-08-31T10:03:02.090Z
Comments.Text: @user172818 minimap2 works amazingly good... many thanks for redirecting me to it
Comments.UserDisplayName: aechchiki
Tags: <sam><rna-alignment><rna-splicing>
Title: Convert local alignments to spliced alignments in SAM file
ThreadText: # Convert local alignments to spliced alignments in SAM file
  
  (Asked by: username_0 on 2017-08-30)
  
  I mapped RNA reads to reference genome, using [LAST](http://last.cbrc.jp/) in [split mode](http://last.cbrc.jp/doc/last-split.html), and converted the MAF alignment to SAM with [maf-convert](http://last.cbrc.jp/doc/maf-convert.html).
  
  My problem is that the transcripts are not reported in a spliced manner, meaning that a `transcript_ID` is reported several times in the alignment SAM file with an identical bitwise flag in `$2`. From what I understand, this is due to the fact that only the exons are mapped (one exon per row) and reported as local alignments, as the software cannot deal with splicing patterns combination exon-intron (yet), which behaviour is clear from the CIGAR string.
  
  For a more concrete and visual example let's consider the mapping of transcript `FBtr0344900` to the reference genome as given by LAST:
  
  ```
  $ cat last_aln.sam | grep FBtr0344900
  FBtr0344900 0   4   42774   100 384=144H    *   0   0   TGCGACATTGTTCTACGATGACTACAAAAAATGACCAATAACTTCTATAAACCAATACGATATGTCAGGAGTTTCGGTCCCATACGAAGTCGCCGACTTAAGTATTTTATttttattttgatATGTGTTTGCTATTTTACCTTGTCGAATGCTTCCACACGCTATGAGAATACCATCGTGAGCGTAGCTTACTACTAGAATTTTGTTGAAGTTATTGACAAGCGATGTCTCAATATCTTCCGGACAGCCTCCAGCGTGACATTGCGGGGAATCATGTAACGGCCCAGTAACAGCCTCGGCCAGCACTCGAAGGTTTTCGTTAAGTTTAAGTATTTTATTTGTAGCACCCGCAAACAAAACATTGTGCATAAAGTCGAAGCTCAT    *   NM:i:0  AS:i:2304
  FBtr0344900 0   4   43231   100 384H144=    *   0   0   CTGGAAGCTGTTGATTGAACTGGTATTGATGGCAAGTTAAACTGGGCGACTATGTCATTTAAGGGAGATAACGCCTGAGCCGGCAGTTCTTCAATGCAGTTAACGCAATAATGCTGAGAACCGAGTATGATAATAATACACAGT    *   NM:i:0  AS:i:864
  
  ```
  
  And here is the mapping of the same transcript `FBtr0344900` as given by [STAR](https://github.com/alexdobin/STAR) - software which reports the alignment right the way I need it:
  
  ```
  cat star_aln.sam | grep FBtr0344900
  FBtr0344900    0    4    42774    255    384M73N144M    *    0    0TGCGACATTGTTCTACGATGACTACAAAAAATGACCAATAACTTCTATAAACCAATACGATATGTCAGGAGTTTCGGTCCCATACGAAGTCGCCGACTTAAGTATTTTATTTTTATTTTGATATGTGTTTGCTATTTTACCTTGTCGAATGCTTCCACACGCTATGAGAATACCATCGTGAGCGTAGCTTACTACTAGAATTTTGTTGAAGTTATTGACAAGCGATGTCTCAATATCTTCCGGACAGCCTCCAGCGTGACATTGCGGGGAATCATGTAACGGCCCAGTAACAGCCTCGGCCAGCACTCGAAGGTTTTCGTTAAGTTTAAGTATTTTATTTGTAGCACCCGCAAACAAAACATTGTGCATAAAGTCGAAGCTCATCTGGAAGCTGTTGATTGAACTGGTATTGATGGCAAGTTAAACTGGGCGACTATGTCATTTAAGGGAGATAACGCCTGAGCCGGCAGTTCTTCAATGCAGTTAACGCAATAATGCTGAGAACCGAGTATGATAATAATACACAGT    *    NH:i:1    HI:i:1    NM:i:0    MD:Z:528
  
  ```
  
  From [discussion](https://groups.google.com/forum/#!topic/last-align/xpseSgE7Avs) with the author, it seems that I cannot get what I need straight from the current LAST version, and is not a technical problem. So, I have to modify the output myself. The aim would be to get at least a CIGAR line representing a complete transcript.
  
  My question is, do you know any software to do this? What I would need is a file containing one line per uniquely mapped transcripts, featuring three fields: `transcript_ID`, `start position` and `CIGAR string`.
  
  From my side, I proceeded like this :
  
  1) extract the fields that I need for the interested in from the SAM file:
  
  ```
  $ cut -f1,4,6
  FBtr0344900  42774  384=144H
  FBtr0344900  43231  384H144=
  
  ```
  
  2) split the CIGAR line to remove items I am not interested in - I simplify the command here, just assuming I only have perfect matches (which I am interested in) and hard-clipping (which I am not interested in):
  
  ```
  $ cut -f3 | sed 's/H/_ /g'| sed 's/=/= /g' | sed 's/\w*_\s*//' | sed 's/=//g'
  384
  144
  
  ```
  
  3) paste the modified CIGAR in the original file, with `paste`, resulting in:
  
  ```
  $ paste (1) (2) | cut -f1,2,4
  FBtr0344900  42774  384
  FBtr0344900  43231  144
  
  ```
  
  4) merge rows which start ba the same `transcript_id`:
  
  ```
  $ awk -F'\t' -v OFS='\t' '{ x=$1 ; $1="" ; a[x]=a[x]$0 } END { for(x in a) print x,a[x] }' | 
  FBtr0344900    42774    384    43231   144 
  
  ```
  
  5) compute the new cigar, calculating the intron length as the arithmetic formula `intron_length= (next_exon_start_coordinate - exon_length - previous_exon_start_coordinate)`, in this simple case above: `intron_length= 43231-384-42774`
  
  ```
  $ awk '{ printf("%s", $1) }; { for (i=4; i<=NF; i+=2) { printf("%s%d%s%d", OFS, $(i-1), OFS, $i - $(i-1) - $(i-2)) } }; { printf("%s%d%s%s", OFS, $NF, OFS, RS) }'
  FBtr0344900 384 73 144
  
  ```
  
  6) ideally, with some method that I will figure out, I will modify the CIGAR string (adding alternate M,N at the end of each field except first one), this is how the final file should look like:
  
  ```
  FBtr0344900 42774  384M73N144M
  
  ```
  
  The issue of my basic approach are that:
  
  1. I am not sure how to account for the 1-based SAM : should I add `+1` at every `exon_start_coordinate`? Does not look like, as the STAR output has the exact same cigar string that I calculated on the STAR output.
  2. BIG issue: This will only work for the reads mapped in forward strand: how to make it feasible with reads mapped in the reverse strand? If I keep my current approach, I will have negative intron sizes...
  
  Any suggestion is welcome!
  
  **Tags:** sam, rna-alignment, rna-splicing
  
  > **Comments:**
  > **username_1**: I strongly encourage you to not do this with normal unix tools and to instead code something in python (or whatever language you prefer).  
  > **username_2**: I would use an aligner that outputs proper SAM, such as star, gmap, spaln or minimap2.  
  > **username_0**: yes, I already run the same alignment with GMAP, STAR. we have suspect that LAST might work quite well, but as for now the output is not directly helpful. I will try minimap2 (nice to know you can use it for RNA also)  
  > **username_0**: @username_2 minimap2 works amazingly good... many thanks for redirecting me to it  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2018-11-18. Score: 3 | Accepted answer
  
  Just use minimap2 in split alignment mode to realign the reads.
  
  If that is not an option, then you could try using [pysam](https://pysam.readthedocs.io/) to modify the CIGAR strings. I do not recommend this, as there are many opportunities for subtle bugs because the SAM spec is complex. You would need to:
  
  1. Sort the BAM on read ID so that you can efficiently retrieve reads that you want to merge
  2. Iterate through the BAM with pysam while loading all reads of the same read ID
  3. For each set of reads with the same read ID:
     4. Order the reads of the same read ID by the number of hard-clipped bases at the start of the CIGAR string.
     5. Build a new CIGAR string from the ordered CIGAR strings
     6. Merge the meta tags from the ordered reads
     7. Write out a single, merged read
  
  > **Comments:**
  > **username_0**: I did that at the end. At the time, I just wanted to reformat the non-spliced alignments from LAST (for example) into spliced alignments. Why? Benchmark the most tools possible. But it's ok, I gave up at the end and just used tools that already gave that spliced alignment directly.  
  > **username_3**: That's great. However, the point of SE is to also help others that might have similar issues in the future by providing answers to questions so that users can upvote good answers. If you feel that my answer is a good one to the question you posed, then please consider up voting and accepting it.  
  > **username_3**: If you feel that this answer does not answer the question you really had, then please consider rewriting your question for clarity.  
  
  ---
  
  