Answers.Body: <p>That isn't actually a bam file <a href="https://bioinformatics.stackexchange.com/a/7444/298">as John Marshall figured out</a>. I am keeping the rest of my answer since it could be useful to someone else, but the issue here was that you had a compressed (bgzipped) sam file and not an actual bam file and that's why you were getting that error. When I sorted your file in preparation for indexing it, I converted to a bam which is why the rest of my answer worked.</p>
  
  <hr>
  
  <p>You don't have the index file for the bam file you're using. I used this script on the file you linked to (changing the name so that it corresponds to the right file and a contig in that file):</p>
  
  <pre><code>#!/usr/bin/env python3
  
  import pysam
  samfile = pysam.AlignmentFile("3_Tce_1_mapped.bam", "rb")
  for read in samfile.fetch('3_Tce_b3v08_scaf005149'):
      print(read)
  samfile.close()
  </code></pre>
  
  <p>The directory I ran it in had:</p>
  
  <pre><code>$ ls 3*
  3_Tce_1_mapped.bam
  </code></pre>
  
  <p>And I got the error you described:</p>
  
  <pre><code>$ foo.py
  Traceback (most recent call last):
    File "/home/terdon/scripts/foo.py", line 5, in &lt;module&gt;
      for read in samfile.fetch('3_Tce_b3v08_scaf005149'):
    File "pysam/libcalignmentfile.pyx", line 1107, in pysam.libcalignmentfile.AlignmentFile.fetch
  ValueError: fetching by region is not available for SAM files
  </code></pre>
  
  <p>However, indexing the bam file fixed it:</p>
  
  <pre class="lang-none prettyprint-override"><code><span class="math-container">$ samtools sort 3_Tce_1_mapped.bam &gt; 3_Tce_1_mapped.sorted.bam 
  $</span> mv 3_Tce_1_mapped.sorted.bam 3_Tce_1_mapped.bam
  <span class="math-container">$ samtools index 3_Tce_1_mapped.bam 
  $</span> ls 3*
  3_Tce_1_mapped.bam  3_Tce_1_mapped.bam.bai
  
  $ foo.py | wc
  227    2724   16725
  </code></pre>
  
  <p>So just sort and index your files before attempting to seek in them. Which makes sense since the index's job is primarily to allow seeking.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-10T16:10:46.163Z
Answers.LastActivityDate: 2019-04-11T09:54:03.857Z
Answers.LastEditDate: 2019-04-11T09:54:03.857Z
Answers.UserDisplayName: terdon
Answers.Body: <p>Your <em>3_Tms_1_mapped.bam</em> file, despite its filename extension, is in fact a bgzipped <strong>SAM</strong> file. You can verify this using <a href="http://www.htslib.org/doc/htsfile.html" rel="nofollow noreferrer"><code>htsfile</code></a>, which is a small utility packaged with HTSlib:</p>
  
  <pre class="lang-none prettyprint-override"><code>$ htsfile 3_Tms_1_mapped.bam 
  3_Tms_1_mapped.bam: SAM version 1.3 BGZF-compressed sequence data
  </code></pre>
  
  <p>(For files that really are in BAM format, it reports <em>BAM version 1 compressed sequence data</em>.)</p>
  
  <p>So the error message is accurate in this case.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-10T21:52:42.710Z
Answers.Text: Hmm, does it mean that this link is wrong? http://biopython.org/DIST/docs/api/Bio.bgzf-module.html I sort of thought that it's the same thing...
Answers.UserDisplayName: Kamil S Jaron
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-10T21:58:40.487Z
Answers.Text: No. (Was there a particular part of that you think might be wrong?) But there is a difference between BGZF-compressing the plain text format, and a BAM file (whose decompressed underlying stream is a bespoke binary format that is different from the plain SAM text).
Answers.UserDisplayName: John Marshall
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-11T07:05:50.743Z
Answers.Text: I think I get it now. I got confused by the aims to this module, they write "In addition to being required for random access to and writing of BAM files", so I got an impression that I can simply use this module to write BAM files. This was actually really helpful answer, thanks!
Answers.UserDisplayName: Kamil S Jaron
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-11T08:10:21.547Z
Answers.Text: @KamilSJaron you might want to consider accepting this answer instead of mine since this was the real issue.
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-10T20:43:55.863Z
Answers.LastActivityDate: 2019-04-11T06:36:38.400Z
Answers.LastEditDate: 2019-04-11T06:36:38.400Z
Answers.UserDisplayName: John Marshall
Body: <p>I am used to <code>gzip</code>/<code>biopython</code> solutions when dealing with sequencing data, but now I wish to switch to more elegant <code>pysam</code>. So I looked at <a href="https://pysam.readthedocs.io/en/latest/api.html" rel="nofollow noreferrer">the manual</a>, but ran into quite bizarre troubles with the first couple of lines using my bam file</p>
  
  <pre><code>import pysam
  samfile = pysam.AlignmentFile("3_Tms_1_mapped.bam", "rb")
  for read in samfile.fetch('3_Tms_b3v08_scaf000159'):
      print(read)
  samfile.close()
  </code></pre>
  
  <p>returns <code>ValueError: fetching by region is not available for SAM files</code>. Well, the file is <code>bam</code>. I tried to google the error but the only hits I found are <a href="https://github.com/pysam-developers/pysam/blob/master/pysam/libcalignmentfile.pyx#L1085-L1109" rel="nofollow noreferrer">the lines in the source code of pysam</a> that check if the file is bam/cram or sam, so somehow <code>pysam</code> thinks that my bam is a sam. How can I convince it otherwise? I have also noticed that the manual is for python 2.7, that's maybe where the problem comes from...</p>
  
Tags: <python><sam><pysam>
Title: What does "fetching by region is not available for SAM files" mean?
ThreadText: # What does "fetching by region is not available for SAM files" mean?
  
  (Asked by: username_0 on 2019-04-10)
  
  I am used to `gzip`/`biopython` solutions when dealing with sequencing data, but now I wish to switch to more elegant `pysam`. So I looked at [the manual](https://pysam.readthedocs.io/en/latest/api.html), but ran into quite bizarre troubles with the first couple of lines using my bam file
  
  ```
  import pysam
  samfile = pysam.AlignmentFile("3_Tms_1_mapped.bam", "rb")
  for read in samfile.fetch('3_Tms_b3v08_scaf000159'):
      print(read)
  samfile.close()
  
  ```
  
  returns `ValueError: fetching by region is not available for SAM files`. Well, the file is `bam`. I tried to google the error but the only hits I found are [the lines in the source code of pysam](https://github.com/pysam-developers/pysam/blob/master/pysam/libcalignmentfile.pyx#L1085-L1109) that check if the file is bam/cram or sam, so somehow `pysam` thinks that my bam is a sam. How can I convince it otherwise? I have also noticed that the manual is for python 2.7, that's maybe where the problem comes from...
  
  **Tags:** python, sam, pysam
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2019-04-10. Score: 5 | Accepted answer
  
  Your *3\_Tms\_1\_mapped.bam* file, despite its filename extension, is in fact a bgzipped **SAM** file. You can verify this using [`htsfile`](http://www.htslib.org/doc/htsfile.html), which is a small utility packaged with HTSlib:
  
  ```none
  $ htsfile 3_Tms_1_mapped.bam 
  3_Tms_1_mapped.bam: SAM version 1.3 BGZF-compressed sequence data
  
  ```
  
  (For files that really are in BAM format, it reports *BAM version 1 compressed sequence data*.)
  
  So the error message is accurate in this case.
  
  > **Comments:**
  > **username_0**: Hmm, does it mean that this link is wrong? http://biopython.org/DIST/docs/api/Bio.bgzf-module.html I sort of thought that it's the same thing...  
  > **username_2**: No. (Was there a particular part of that you think might be wrong?) But there is a difference between BGZF-compressing the plain text format, and a BAM file (whose decompressed underlying stream is a bespoke binary format that is different from the plain SAM text).  
  > **username_0**: I think I get it now. I got confused by the aims to this module, they write "In addition to being required for random access to and writing of BAM files", so I got an impression that I can simply use this module to write BAM files. This was actually really helpful answer, thanks!  
  > **username_1**: @username_0 you might want to consider accepting this answer instead of mine since this was the real issue.  
  
  ---
  
  ### Answer by username_1 on 2019-04-10. Score: 5
  
  That isn't actually a bam file [as username_2 username_2 figured out](https://bioinformatics.stackexchange.com/a/7444/298). I am keeping the rest of my answer since it could be useful to someone else, but the issue here was that you had a compressed (bgzipped) sam file and not an actual bam file and that's why you were getting that error. When I sorted your file in preparation for indexing it, I converted to a bam which is why the rest of my answer worked.
  
  ---
  
  You don't have the index file for the bam file you're using. I used this script on the file you linked to (changing the name so that it corresponds to the right file and a contig in that file):
  
  ```
  #!/usr/bin/env python3
  
  import pysam
  samfile = pysam.AlignmentFile("3_Tce_1_mapped.bam", "rb")
  for read in samfile.fetch('3_Tce_b3v08_scaf005149'):
      print(read)
  samfile.close()
  
  ```
  
  The directory I ran it in had:
  
  ```
  $ ls 3*
  3_Tce_1_mapped.bam
  
  ```
  
  And I got the error you described:
  
  ```
  $ foo.py
  Traceback (most recent call last):
    File "/home/username_1/scripts/foo.py", line 5, in <module>
      for read in samfile.fetch('3_Tce_b3v08_scaf005149'):
    File "pysam/libcalignmentfile.pyx", line 1107, in pysam.libcalignmentfile.AlignmentFile.fetch
  ValueError: fetching by region is not available for SAM files
  
  ```
  
  However, indexing the bam file fixed it:
  
  ```none
  $ samtools sort 3_Tce_1_mapped.bam > 3_Tce_1_mapped.sorted.bam 
  $ mv 3_Tce_1_mapped.sorted.bam 3_Tce_1_mapped.bam
  $ samtools index 3_Tce_1_mapped.bam 
  $ ls 3*
  3_Tce_1_mapped.bam  3_Tce_1_mapped.bam.bai
  
  $ foo.py | wc
  227    2724   16725
  
  ```
  
  So just sort and index your files before attempting to seek in them. Which makes sense since the index's job is primarily to allow seeking.
  
  
  ---
  
  