Answers.Body: <p><code>samtools quickcheck</code> is all you need. From the manual:</p>
  <blockquote>
  <p>Quickly check that input files appear to be intact. Checks that beginning of the file contains a valid header (all formats) containing at least one target sequence and then seeks to the end of the file and checks that an end-of-file (EOF) is present and intact (BAM only).</p>
  <p>Data in the middle of the file is not read since that would be much more time consuming, so please note that this command will not detect internal corruption, but is useful for testing that files are not truncated before performing more intensive tasks on them.</p>
  <p>This command will exit with a non-zero exit code if any input files don't have a valid header or are missing an EOF block. Otherwise it will exit successfully (with a zero exit code).</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-05-01T03:23:32.533Z
Answers.Text: Just a note if your cluster has older modules, or old binaries in your search path: `samtools quickcheck` requires samtools ≥ 1.3. For older versions you can `samtools stats BAMFILE |& grep -q 'EOF marker is absent' && whatever`, but this still tries to compute stats for the non-corrupted parts of the BAM so it's not the fastest solution.
Answers.UserDisplayName: Kevin E
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-08T21:05:11.887Z
Answers.LastActivityDate: 2018-05-08T21:05:11.887Z
Answers.LastEditDate: 2020-06-18T08:30:41.743Z
Answers.UserDisplayName: Devon Ryan
Body: <p>I currently have several hundred BAM files which were downloaded by someone else. These have remained untouched---before working with them, I would like to double-check that these BAMs have been fully downloaded. </p>
  
  <p>I don't MD5 Checksums to look at. </p>
  
  <p>In order to validate the BAMs, I would use something like Picard's ValidateSamFile, <a href="https://broadinstitute.github.io/picard/command-line-overview.html#ValidateSamFile" rel="noreferrer">https://broadinstitute.github.io/picard/command-line-overview.html#ValidateSamFile</a></p>
  
  <pre><code>java -jar picard.jar ValidateSamFile \
        I=input.bam \
        MODE=SUMMARY
  </code></pre>
  
  <p>I could write a bash script, and run through these BAMs, and then read through the Picard output. That is checking whether the BAM is valid though, not whether it was downloaded fully (e.g. the BAM could be downloaded correctly, but be inherently flawed in some other way).</p>
  
  <p>If the BAM is downloaded fully, isn't there some "end of file" bytes I could quickly check? Are there other more efficient methods?</p>
  
Tags: <sam><data-download><picard><validation>
Title: How to validate that BAMs have been downloaded correctly?
ThreadText: # How to validate that BAMs have been downloaded correctly?
  
  (Asked by: username_0 on 2018-05-08)
  
  I currently have several hundred BAM files which were downloaded by someone else. These have remained untouched---before working with them, I would like to double-check that these BAMs have been fully downloaded.
  
  I don't MD5 Checksums to look at.
  
  In order to validate the BAMs, I would use something like Picard's ValidateSamFile, <https://broadinstitute.github.io/picard/command-line-overview.html#ValidateSamFile>
  
  ```
  java -jar picard.jar ValidateSamFile \
        I=input.bam \
        MODE=SUMMARY
  
  ```
  
  I could write a bash script, and run through these BAMs, and then read through the Picard output. That is checking whether the BAM is valid though, not whether it was downloaded fully (e.g. the BAM could be downloaded correctly, but be inherently flawed in some other way).
  
  If the BAM is downloaded fully, isn't there some "end of file" bytes I could quickly check? Are there other more efficient methods?
  
  **Tags:** sam, data-download, picard, validation
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-05-08. Score: 9 | Accepted answer
  
  `samtools quickcheck` is all you need. From the manual:
  
  > Quickly check that input files appear to be intact. Checks that beginning of the file contains a valid header (all formats) containing at least one target sequence and then seeks to the end of the file and checks that an end-of-file (EOF) is present and intact (BAM only).
  >
  > Data in the middle of the file is not read since that would be much more time consuming, so please note that this command will not detect internal corruption, but is useful for testing that files are not truncated before performing more intensive tasks on them.
  >
  > This command will exit with a non-zero exit code if any input files don't have a valid header or are missing an EOF block. Otherwise it will exit successfully (with a zero exit code).
  
  > **Comments:**
  > **username_2**: Just a note if your cluster has older modules, or old binaries in your search path: `samtools quickcheck` requires samtools ≥ 1.3. For older versions you can `samtools stats BAMFILE |& grep -q 'EOF marker is absent' && whatever`, but this still tries to compute stats for the non-corrupted parts of the BAM so it's not the fastest solution.  
  
  ---
  
  