Answers.Body: <blockquote>
  <p>Alternatively, maybe generate genotype likelihood files in small batches then merge? It does run successfully on 30 of the 122 samples.</p>
  </blockquote>
  <p>Yes, that would be my recommendation if you're having trouble with memory. The standard strategy recommended in GATK is similar to this, creating genotype likelihoods individually, then merging for genotyping:</p>
  <p><a href="https://github.com/broadinstitute/gatk-docs/blob/master/gatk3-methods-and-algorithms/Best_Practices_for_Variant_Discovery_in_DNAseq.md" rel="nofollow noreferrer">https://github.com/broadinstitute/gatk-docs/blob/master/gatk3-methods-and-algorithms/Best_Practices_for_Variant_Discovery_in_DNAseq.md</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-30T09:00:17.887Z
Answers.LastActivityDate: 2024-12-30T09:00:17.887Z
Answers.UserDisplayName: gringer
Body: <p>I am currently trying to run angsd v0.940 on a whole genome shotgun sequence dataset (122 samples; avg file size: 13G, min: 4G, max: 34G) but it keeps running out memory no matter how I try to tune the parameters. For instance, I tried to change the number of threads from very high values like 75 threads to 2 or 1 following suggestions here: <a href="https://github.com/ANGSD/angsd/issues/243" rel="nofollow noreferrer">https://github.com/ANGSD/angsd/issues/243</a>). My reference genome size is 1GB and 58,006 contigs. I'm trying to generate a genotype likelihood dataset but I can't think of any other way. Any thoughts would be appreciated!</p>
  <p>My command:</p>
  <pre><code>angsd -bam sample_bam_list -GL 1 -doMajorMinor 1 -doMaf 1 -doGlf 2 -minMaf 0.05 -SNP_pval 0.01 -minInd 122 -minMapQ 20 -minQ 20 -nThreads 1 -out cbth
  </code></pre>
  <p>Any suggestions would be greatly appreciated. Maybe there is a way to break up the bam files into smaller sizes. Alternatively, maybe generate genotype liklihood files in small batches then merge? It does run successfully on 30 of the 122 samples.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-12-30T02:53:31.287Z
Comments.Text: What machine are you running it on? Do you have access to a AWS or GCP instance?
Comments.UserDisplayName: M__
Tags: <sam><genome><samtools><variant-calling><variants>
Title: strategy for generating a genotype likelihood file from large bam files and many samples
ThreadText: # strategy for generating a genotype likelihood file from large bam files and many samples
  
  (Asked by: username_0 on 2024-12-29)
  
  I am currently trying to run angsd v0.940 on a whole genome shotgun sequence dataset (122 samples; avg file size: 13G, min: 4G, max: 34G) but it keeps running out memory no matter how I try to tune the parameters. For instance, I tried to change the number of threads from very high values like 75 threads to 2 or 1 following suggestions here: <https://github.com/ANGSD/angsd/issues/243>). My reference genome size is 1GB and 58,006 contigs. I'm trying to generate a genotype likelihood dataset but I can't think of any other way. Any thoughts would be appreciated!
  
  My command:
  
  ```
  angsd -bam sample_bam_list -GL 1 -doMajorMinor 1 -doMaf 1 -doGlf 2 -minMaf 0.05 -SNP_pval 0.01 -minInd 122 -minMapQ 20 -minQ 20 -nThreads 1 -out cbth
  
  ```
  
  Any suggestions would be greatly appreciated. Maybe there is a way to break up the bam files into smaller sizes. Alternatively, maybe generate genotype liklihood files in small batches then merge? It does run successfully on 30 of the 122 samples.
  
  **Tags:** sam, genome, samtools, variant-calling, variants
  
  > **Comments:**
  > **username_1**: What machine are you running it on? Do you have access to a AWS or GCP instance?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2024-12-30. Score: 0
  
  > Alternatively, maybe generate genotype likelihood files in small batches then merge? It does run successfully on 30 of the 122 samples.
  
  Yes, that would be my recommendation if you're having trouble with memory. The standard strategy recommended in GATK is similar to this, creating genotype likelihoods individually, then merging for genotyping:
  
  <https://github.com/broadinstitute/gatk-docs/blob/master/gatk3-methods-and-algorithms/Best_Practices_for_Variant_Discovery_in_DNAseq.md>
  
  
  ---
  
  