Answers.Body: <p><code>CollectInsertSizeMetrics</code> does not estimate but (as by the name) <strong>collects</strong> insert size metrics, which is nothing different than parsing the TLEN field from the BAM file. It is intended to only be used for paired-end data, not single-end, as single-end data do not have a value in that field in the BAM file. It is calculated by the aligner for paired-end data.</p>
  <p>What kind of data do you have? For targeted assays such as ChIP-seq the <code>macs2</code> peak caller has a module to estimate fragment size based on strand cross correlation analysis. I do not kow whether you can use that for any sort of assay.</p>
  <p>If this is your own data then you can ask the lab person who made the library to look up the size from the Bioanalyzer or TapeStation QC. If it is published data then check the methods text for the procedure/kit. Most NGS types have a relatively standard insert size depending on the exact assay.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-31T18:02:01.870Z
Answers.Text: It is from published data (RNA-Seq). I want to do alternative splicing analysis with MISO, which asks for the insert size. 
  Thanks for your suggestions, I will look deeper into the methods.
Answers.UserDisplayName: justinian482
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-31T19:03:49.007Z
Answers.Text: Can you link the GEO reference? RNA-seq libraries usually have a somewhatish 200-300bp insert size.
Answers.UserDisplayName: ATpoint
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-01T08:18:53.060Z
Answers.Text: https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE121949
Answers.UserDisplayName: justinian482
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-31T17:40:55.367Z
Answers.LastActivityDate: 2023-01-31T17:40:55.367Z
Answers.UserDisplayName: ATpoint
Body: <p>I have a BAM file generated from single-end data and I want to estimate the insert size using picard's <code>CollectInsertSizeMetrics</code> as follows:</p>
  <pre><code>java -jar Picard/picard.jar CollectInsertSizeMetrics I=Ctrl_sorted.bam O=Ctrl.insert_sizes.txt H=Ctrl.insert_sizes.pdf M=0.5 MAX_RECORDS_IN_RAM=2000000 VALIDATION_STRINGENCY=LENIENT
  </code></pre>
  <p>It runs fine until at the very end I get the following message:</p>
  <pre><code>WARNING 2023-01-31 15:37:13 CollectInsertSizeMetrics    All data categories were discarded because they contained &lt; 0.5 of the total aligned paired data.
  
  </code></pre>
  <p>Picard obviously thinks the data should be paired-end, so is there a way to tell it that it is single-end? I cannot find an option for this:</p>
  <pre><code>java -jar picard.jar CollectInsertSizeMetrics -h
  USAGE: CollectInsertSizeMetrics [arguments]
  
  Collect metrics about the insert size distribution of a paired-end library. This tool provides useful metrics for
  validating library construction including the insert size distribution and read orientation of paired-end
  libraries.&lt;/p&gt;The expected proportions of these metrics vary depending on the type of library preparation used,
  resulting from technical differences between pair-end libraries and mate-pair libraries. For a brief primer on
  paired-end sequencing and mate-pair reads, see the &lt;a
  href='https://www.broadinstitute.org/gatk/guide/article?id=6327'&gt;GATK Dictionary&lt;/a&gt;.&lt;p&gt;The CollectInsertSizeMetrics
  tool outputs the percentages of read pairs in each of the three orientations (FR, RF, and TANDEM) as a histogram. In
  addition, the insert size distribution is output as both a histogram (.insert_size_Histogram.pdf) and as a data table
  (.insert_size_metrics.txt).&lt;/p&gt;&lt;p&gt;Note: Metrics labeled as percentages are actually expressed as fractions!&lt;/p&gt;&lt;h4&gt;Usage
  example:&lt;/h4&gt;&lt;pre&gt;java -jar picard.jar CollectInsertSizeMetrics \&lt;br /&gt;      I=input.bam \&lt;br /&gt;     
  O=insert_size_metrics.txt \&lt;br /&gt;      H=insert_size_histogram.pdf \&lt;br /&gt;      M=0.5&lt;/pre&gt;Note: If processing a small
  file, set the minimum percentage option (M) to 0.5, otherwise an error may occur. &lt;br /&gt;&lt;br /&gt;Please see &lt;a
  href='https://broadinstitute.github.io/picard/picard-metric-definitions.html#InsertSizeMetrics'&gt;InsertSizeMetrics&lt;/a&gt;
  for detailed explanations of each metric.&lt;hr /&gt;
  Version:2.23.8
  
  
  Required Arguments:
  
  --Histogram_FILE,-H:File      File to write insert size Histogram chart to.  Required. 
  
  --INPUT,-I:File               Input SAM or BAM file.  Required. 
  
  --OUTPUT,-O:File              The file to write the output to.  Required. 
  
  
  Optional Arguments:
  
  --arguments_file:File         read one or more arguments files and add them to the command line  This argument may be
                                specified 0 or more times. Default value: null. 
  
  --ASSUME_SORTED,-AS:Boolean   If true (default), then the sort order in the header file will be ignored.  Default value:
                                true. Possible values: {true, false} 
  
  --COMPRESSION_LEVEL:Integer   Compression level for all compressed files created (e.g. BAM and VCF).  Default value: 5. 
  
  --CREATE_INDEX:Boolean        Whether to create an index when writing VCF or coordinate sorted BAM output.  Default
                                value: false. Possible values: {true, false} 
  
  --CREATE_MD5_FILE:Boolean     Whether to create an MD5 digest for any BAM or FASTQ files created.    Default value:
                                false. Possible values: {true, false} 
  
  --DEVIATIONS:Double           Generate mean, sd and plots by trimming the data down to MEDIAN +
                                DEVIATIONS*MEDIAN_ABSOLUTE_DEVIATION. This is done because insert size data typically
                                includes enough anomalous values from chimeras and other artifacts to make the mean and sd
                                grossly misleading regarding the real distribution.  Default value: 10.0. 
  
  --GA4GH_CLIENT_SECRETS:String Google Genomics API client_secrets.json file path.  Default value: client_secrets.json. 
  
  --help,-h:Boolean             display the help message  Default value: false. Possible values: {true, false} 
  
  --HISTOGRAM_WIDTH,-W:Integer  Explicitly sets the Histogram width, overriding automatic truncation of Histogram tail.
                                Also, when calculating mean and standard deviation, only bins &lt;= Histogram_WIDTH will be
                                included.  Default value: null. 
  
  --INCLUDE_DUPLICATES:Boolean  If true, also include reads marked as duplicates in the insert size histogram.  Default
                                value: false. Possible values: {true, false} 
  
  --MAX_RECORDS_IN_RAM:Integer  When writing files that need to be sorted, this will specify the number of records stored
                                in RAM before spilling to disk. Increasing this number reduces the number of file handles
                                needed to sort the file, and increases the amount of RAM needed.  Default value: 500000. 
  
  --METRIC_ACCUMULATION_LEVEL,-LEVEL:MetricAccumulationLevel
                                The level(s) at which to accumulate metrics.    This argument may be specified 0 or more
                                times. Default value: [ALL_READS]. Possible values: {ALL_READS, SAMPLE, LIBRARY,
                                READ_GROUP} 
  
  --MIN_HISTOGRAM_WIDTH,-MW:Integer
                                Minimum width of histogram plots. In the case when the histogram would otherwise
                                betruncated to a shorter range of sizes, the MIN_HISTOGRAM_WIDTH will enforce a minimum
                                range.  Default value: null. 
  
  --MINIMUM_PCT,-M:Float        When generating the Histogram, discard any data categories (out of FR, TANDEM, RF) that
                                have fewer than this percentage of overall reads. (Range: 0 to 1).  Default value: 0.05. 
  
  --QUIET:Boolean               Whether to suppress job-summary info on System.err.  Default value: false. Possible
                                values: {true, false} 
  
  --REFERENCE_SEQUENCE,-R:File  Reference sequence file.  Default value: null. 
  
  --STOP_AFTER:Long             Stop after processing N reads, mainly for debugging.  Default value: 0. 
  
  --TMP_DIR:File                One or more directories with space available to be used by this program for temporary
                                storage of working files  This argument may be specified 0 or more times. Default value:
                                null. 
  
  --USE_JDK_DEFLATER,-use_jdk_deflater:Boolean
                                Use the JDK Deflater instead of the Intel Deflater for writing compressed output  Default
                                value: false. Possible values: {true, false} 
  
  --USE_JDK_INFLATER,-use_jdk_inflater:Boolean
                                Use the JDK Inflater instead of the Intel Inflater for reading compressed input  Default
                                value: false. Possible values: {true, false} 
  
  --VALIDATION_STRINGENCY:ValidationStringency
                                Validation stringency for all SAM files read by this program.  Setting stringency to
                                SILENT can improve performance when processing a BAM file in which variable-length data
                                (read, qualities, tags) do not otherwise need to be decoded.  Default value: STRICT.
                                Possible values: {STRICT, LENIENT, SILENT} 
  
  --VERBOSITY:LogLevel          Control verbosity of logging.  Default value: INFO. Possible values: {ERROR, WARNING,
                                INFO, DEBUG} 
  
  --version:Boolean             display the version number for this tool  Default value: false. Possible values: {true,
                                false} 
  
  
  Advanced Arguments:
  
  --showHidden,-showHidden:Boolean
                                display hidden arguments  Default value: false. Possible values: {true, false} 
  
  </code></pre>
  <p>Or is there another tool that can deal with single-end data to estimate the insert size?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-02T19:12:51.567Z
Comments.Text: I can't see how you would calculate insert size from single end data. Are you aware of a paper about it, or something like that?
Comments.UserDisplayName: gringer
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-02T22:40:53.487Z
Comments.Text: I just realised I wasn't thinking clearly; you are right - from single-end data you cannot calculate the insert size. This can only be established experimentally. Thanks everyone for your comments.
Comments.UserDisplayName: justinian482
Tags: <ngs><sam><picard>
Title: Estimate insert size for single-end data with picard CollectInsertSizeMetrics
ThreadText: # Estimate insert size for single-end data with picard CollectInsertSizeMetrics
  
  (Asked by: username_0 on 2023-01-31)
  
  I have a BAM file generated from single-end data and I want to estimate the insert size using picard's `CollectInsertSizeMetrics` as follows:
  
  ```
  java -jar Picard/picard.jar CollectInsertSizeMetrics I=Ctrl_sorted.bam O=Ctrl.insert_sizes.txt H=Ctrl.insert_sizes.pdf M=0.5 MAX_RECORDS_IN_RAM=2000000 VALIDATION_STRINGENCY=LENIENT
  
  ```
  
  It runs fine until at the very end I get the following message:
  
  ```
  WARNING 2023-01-31 15:37:13 CollectInsertSizeMetrics    All data categories were discarded because they contained < 0.5 of the total aligned paired data.
  
  ```
  
  Picard obviously thinks the data should be paired-end, so is there a way to tell it that it is single-end? I cannot find an option for this:
  
  ```
  java -jar picard.jar CollectInsertSizeMetrics -h
  USAGE: CollectInsertSizeMetrics [arguments]
  
  Collect metrics about the insert size distribution of a paired-end library. This tool provides useful metrics for
  validating library construction including the insert size distribution and read orientation of paired-end
  libraries.</p>The expected proportions of these metrics vary depending on the type of library preparation used,
  resulting from technical differences between pair-end libraries and mate-pair libraries. For a brief primer on
  paired-end sequencing and mate-pair reads, see the <a
  href='https://www.broadinstitute.org/gatk/guide/article?id=6327'>GATK Dictionary</a>.<p>The CollectInsertSizeMetrics
  tool outputs the percentages of read pairs in each of the three orientations (FR, RF, and TANDEM) as a histogram. In
  addition, the insert size distribution is output as both a histogram (.insert_size_Histogram.pdf) and as a data table
  (.insert_size_metrics.txt).</p><p>Note: Metrics labeled as percentages are actually expressed as fractions!</p><h4>Usage
  example:</h4><pre>java -jar picard.jar CollectInsertSizeMetrics \<br />      I=input.bam \<br />     
  O=insert_size_metrics.txt \<br />      H=insert_size_histogram.pdf \<br />      M=0.5</pre>Note: If processing a small
  file, set the minimum percentage option (M) to 0.5, otherwise an error may occur. <br /><br />Please see <a
  href='https://broadinstitute.github.io/picard/picard-metric-definitions.html#InsertSizeMetrics'>InsertSizeMetrics</a>
  for detailed explanations of each metric.<hr />
  Version:2.23.8
  
  Required Arguments:
  
  --Histogram_FILE,-H:File      File to write insert size Histogram chart to.  Required. 
  
  --INPUT,-I:File               Input SAM or BAM file.  Required. 
  
  --OUTPUT,-O:File              The file to write the output to.  Required. 
  
  Optional Arguments:
  
  --arguments_file:File         read one or more arguments files and add them to the command line  This argument may be
                                specified 0 or more times. Default value: null. 
  
  --ASSUME_SORTED,-AS:Boolean   If true (default), then the sort order in the header file will be ignored.  Default value:
                                true. Possible values: {true, false} 
  
  --COMPRESSION_LEVEL:Integer   Compression level for all compressed files created (e.g. BAM and VCF).  Default value: 5. 
  
  --CREATE_INDEX:Boolean        Whether to create an index when writing VCF or coordinate sorted BAM output.  Default
                                value: false. Possible values: {true, false} 
  
  --CREATE_MD5_FILE:Boolean     Whether to create an MD5 digest for any BAM or FASTQ files created.    Default value:
                                false. Possible values: {true, false} 
  
  --DEVIATIONS:Double           Generate mean, sd and plots by trimming the data down to MEDIAN +
                                DEVIATIONS*MEDIAN_ABSOLUTE_DEVIATION. This is done because insert size data typically
                                includes enough anomalous values from chimeras and other artifacts to make the mean and sd
                                grossly misleading regarding the real distribution.  Default value: 10.0. 
  
  --GA4GH_CLIENT_SECRETS:String Google Genomics API client_secrets.json file path.  Default value: client_secrets.json. 
  
  --help,-h:Boolean             display the help message  Default value: false. Possible values: {true, false} 
  
  --HISTOGRAM_WIDTH,-W:Integer  Explicitly sets the Histogram width, overriding automatic truncation of Histogram tail.
                                Also, when calculating mean and standard deviation, only bins <= Histogram_WIDTH will be
                                included.  Default value: null. 
  
  --INCLUDE_DUPLICATES:Boolean  If true, also include reads marked as duplicates in the insert size histogram.  Default
                                value: false. Possible values: {true, false} 
  
  --MAX_RECORDS_IN_RAM:Integer  When writing files that need to be sorted, this will specify the number of records stored
                                in RAM before spilling to disk. Increasing this number reduces the number of file handles
                                needed to sort the file, and increases the amount of RAM needed.  Default value: 500000. 
  
  --METRIC_ACCUMULATION_LEVEL,-LEVEL:MetricAccumulationLevel
                                The level(s) at which to accumulate metrics.    This argument may be specified 0 or more
                                times. Default value: [ALL_READS]. Possible values: {ALL_READS, SAMPLE, LIBRARY,
                                READ_GROUP} 
  
  --MIN_HISTOGRAM_WIDTH,-MW:Integer
                                Minimum width of histogram plots. In the case when the histogram would otherwise
                                betruncated to a shorter range of sizes, the MIN_HISTOGRAM_WIDTH will enforce a minimum
                                range.  Default value: null. 
  
  --MINIMUM_PCT,-M:Float        When generating the Histogram, discard any data categories (out of FR, TANDEM, RF) that
                                have fewer than this percentage of overall reads. (Range: 0 to 1).  Default value: 0.05. 
  
  --QUIET:Boolean               Whether to suppress job-summary info on System.err.  Default value: false. Possible
                                values: {true, false} 
  
  --REFERENCE_SEQUENCE,-R:File  Reference sequence file.  Default value: null. 
  
  --STOP_AFTER:Long             Stop after processing N reads, mainly for debugging.  Default value: 0. 
  
  --TMP_DIR:File                One or more directories with space available to be used by this program for temporary
                                storage of working files  This argument may be specified 0 or more times. Default value:
                                null. 
  
  --USE_JDK_DEFLATER,-use_jdk_deflater:Boolean
                                Use the JDK Deflater instead of the Intel Deflater for writing compressed output  Default
                                value: false. Possible values: {true, false} 
  
  --USE_JDK_INFLATER,-use_jdk_inflater:Boolean
                                Use the JDK Inflater instead of the Intel Inflater for reading compressed input  Default
                                value: false. Possible values: {true, false} 
  
  --VALIDATION_STRINGENCY:ValidationStringency
                                Validation stringency for all SAM files read by this program.  Setting stringency to
                                SILENT can improve performance when processing a BAM file in which variable-length data
                                (read, qualities, tags) do not otherwise need to be decoded.  Default value: STRICT.
                                Possible values: {STRICT, LENIENT, SILENT} 
  
  --VERBOSITY:LogLevel          Control verbosity of logging.  Default value: INFO. Possible values: {ERROR, WARNING,
                                INFO, DEBUG} 
  
  --version:Boolean             display the version number for this tool  Default value: false. Possible values: {true,
                                false} 
  
  Advanced Arguments:
  
  --showHidden,-showHidden:Boolean
                                display hidden arguments  Default value: false. Possible values: {true, false} 
  
  ```
  
  Or is there another tool that can deal with single-end data to estimate the insert size?
  
  **Tags:** ngs, sam, picard
  
  > **Comments:**
  > **username_1**: I can't see how you would calculate insert size from single end data. Are you aware of a paper about it, or something like that?  
  > **username_0**: I just realised I wasn't thinking clearly; you are right - from single-end data you cannot calculate the insert size. This can only be established experimentally. Thanks everyone for your comments.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-01-31. Score: 5 | Accepted answer
  
  `CollectInsertSizeMetrics` does not estimate but (as by the name) **collects** insert size metrics, which is nothing different than parsing the TLEN field from the BAM file. It is intended to only be used for paired-end data, not single-end, as single-end data do not have a value in that field in the BAM file. It is calculated by the aligner for paired-end data.
  
  What kind of data do you have? For targeted assays such as ChIP-seq the `macs2` peak caller has a module to estimate fragment size based on strand cross correlation analysis. I do not kow whether you can use that for any sort of assay.
  
  If this is your own data then you can ask the lab person who made the library to look up the size from the Bioanalyzer or TapeStation QC. If it is published data then check the methods text for the procedure/kit. Most NGS types have a relatively standard insert size depending on the exact assay.
  
  > **Comments:**
  > **username_0**: It is from published data (RNA-Seq). I want to do alternative splicing analysis with MISO, which asks for the insert size.  Thanks for your suggestions, I will look deeper into the methods.  
  > **username_2**: Can you link the GEO reference? RNA-seq libraries usually have a somewhatish 200-300bp insert size.  
  > **username_0**: https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE121949  
  
  ---
  
  