Answers.Body: <p>In addition to the other answer (to reduce the number of threads), you might also need to add a <code>--genomeChrBinNbits</code> argument. Your reference has more than 5000 scaffolds, so you need to adjust the <code>--genomeChrBinNbits</code> argument to reduce RAM consumption. Found <a href="https://github.com/alexdobin/STAR/issues/103" rel="nofollow noreferrer">here</a>.</p>
  
  <p>They recommend:</p>
  
  <blockquote>
    <p>--genomeChrBinNbits} = min(18, log2(GenomeLength/NumberOfReferences))</p>
  </blockquote>
  
  <p>In your case:</p>
  
  <p>Genome size 17G, number of scaffolds 735945</p>
  
  <p>So <code>--genomeChrBinNbits</code> should be set to log2(17,000,000,000/735,945) = 14.4955, so let's say 14 or 15.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-04T16:13:42.720Z
Answers.Text: After your command it gave error to increase ram using limitGenomeGenerateRAM, then I increased the ram using limitGenomeGenerateRAM and finally it worked fine, thanks.
Answers.UserDisplayName: Ammar Sabir Cheema
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-04T16:27:38.393Z
Answers.Text: @AmmarSabirCheema, I am glad it finally works! Thanks for your feedback.
Answers.UserDisplayName: benn
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-04T10:15:30.050Z
Answers.LastActivityDate: 2018-06-04T13:59:39.147Z
Answers.LastEditDate: 2018-06-04T13:59:39.147Z
Answers.UserDisplayName: benn
Answers.Body: <p>Try reducing the number of threads.</p>
  
  <p>When multithreading the index creation (and other memory bound tasks), the memory usage increases linearly with the number of threads. If you required a 10 GiB working space work with a single thread, then the requirement for 10 threads would be at 100 GiB.</p>
  
  <p>STAR is particularly memory hungry for the index creation to start with, and your genome is exceptionally large. I’ve had STAR require > 32 GiB of memory for the human genome, with a moderate number of threads. I’m not surprised that it fails with a 4× larger genome and many more threads: it would probably require at least 2× as much RAM as your machine has.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-04T09:26:10.307Z
Answers.Text: Thanks for your answer but it didn't work.
Answers.UserDisplayName: Ammar Sabir Cheema
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-04T09:27:44.517Z
Answers.Text: I have again edited my post to add more details to my problem.
Answers.UserDisplayName: Ammar Sabir Cheema
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-24T09:32:27.990Z
Answers.LastActivityDate: 2018-05-24T09:32:27.990Z
Answers.UserDisplayName: Konrad Rudolph
Body: <p>I am trying to index wheat genome using STAR through following command</p>
  
  <pre><code>STAR --runMode genomeGenerate --genomeFastaFiles Triticum_aestivum.TGACv1.dna_sm.toplevel.fa --runThreadN 28 
  </code></pre>
  
  <p>But getting following error,</p>
  
  <pre><code>terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
  Aborted (core dumped)
  </code></pre>
  
  <p>The size of the genome is 13.5 GB and the server capacity is 32 cores with 125 GB RAM, how this error can be resolved?</p>
  
  <p>Memory usage: <a href="https://i.sstatic.net/uKcEQ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/uKcEQ.png" alt="enter image description here"></a></p>
  
  <p>I have decreased the number of threads to 8 but again getting the same error as shown below:</p>
  
  <pre><code>&gt; STAR   --runMode genomeGenerate   --runThreadN 8   --genomeDir wheat   --genomeFastaFiles wheat.fa                                                                      
  Jun 04 02:13:31 ..... started STAR run                                                                                                                                                                                           
  Jun 04 02:13:31 ... starting to generate Genome files                                                                                                                                                                            
  terminate called after throwing an instance of 'std::bad_alloc'                                                                                                                                                                  
  what():  std::bad_alloc                                                                                                                                                                                                        
  Aborted (core dumped)                                           
  </code></pre>
  
  <p>Even I have tried to use STAR without any threads but still the same error:</p>
  
  <pre><code>&gt; STAR   --runMode genomeGenerate   --genomeDir wheat   --genomeFastaFiles wheat.fa                                                                         
  Jun 04 02:19:44 ..... started STAR run                                                                                                                                                                                           
  Jun 04 02:19:44 ... starting to generate Genome files                                                                                                                                                                            
  terminate called after throwing an instance of 'std::bad_alloc'                                                                                                                                                                  
  what():  std::bad_alloc
  Aborted (core dumped)
  </code></pre>
  
  <p>May there can be an issue in reference of wheat that I am using, I got this reference from <a href="ftp://ftp.ensemblgenomes.org/pub/plants/release-39/fasta/triticum_aestivum/dna/Triticum_aestivum.TGACv1.dna.toplevel.fa.gz" rel="nofollow noreferrer">here.</a>
  Here are the first ten lines of the reference using head command:</p>
  
  <pre><code>&gt; head wheat.fa
  &gt;Pt dna:chromosome chromosome:TGACv1:Pt:1:135835:1 REF                                                                                                                                                                           
  ACAGAAATACCCAATATCTTGTTCTAGCAAGATATTGGGTATTTCTGTCTTTTCTTTCTT                                                                                                                                                                     
  CAAAAATTCTTATATGTTAGCAGAAAAACCTTATCCATTAATAGATGGAACTTCAACAGC                                                                                                                                                                     
  AGCTAAGTCTAGAGGGAAGTTGTGAGCATTACGTTCGTGCATTACTTCCATACCAAGGTT                                                                                                                                                                     
  AGCACGGTTGATGATATCAGCCCAAGTATTAATAACGCGACCTTGACTATCAACTACAGA                                                                                                                                                                     
  TTGGTTGAAATTGAAACCATTTAGGTTGAAAGCCATAGTACTAATACCTAAAGCAGTGAA                                                                                                                                                                     
  CCAGATTCCTACTACAGGCCAAGCAGCCAAGAAGAAGTGTAAAGAACGAGAGTTGTTGAA                                                                                                                                                                     
  ACTAGCATATTGGAAGATTAATCGGCCAAAATAACCATGAGCAGCCACAATATTATAAGT                                                                                                                                                                     
  TTCTTCCTCTTGACCAAATTTGTAACCCTCATTAGCAGATTCATTTTCAGTAGTTTCCCT                                                                                                                                                                     
  GATCAAACTAGAGGTTACCAAGGAACCATGCATAGCACTGAATAGGGAACCGCCGAATAC 
  </code></pre>
  
  <p>How this problem can be resolved? </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-05-23T09:48:27.167Z
Comments.Text: Keep an eye on the memory usage and see if you're actually exceeding what's available.
Comments.UserDisplayName: Devon Ryan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-05-23T09:58:17.067Z
Comments.Text: The memory does not exceeds 1.6 usage that is normal,
Comments.UserDisplayName: Ammar Sabir Cheema
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-05-23T10:14:05.177Z
Comments.Text: Here is the [link](https://imgur.com/gallery/lRyo3vW) to the memory usage
Comments.UserDisplayName: Ammar Sabir Cheema
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-05-23T11:51:17.537Z
Comments.Text: Try removing the `--sjdbGTF...` stuff from your command as it's likely there's an issue either with the GFF3 file or the fasta file.
Comments.UserDisplayName: Devon Ryan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-05-23T13:16:03.457Z
Comments.Text: Use (much!) fewer threads. Threads speed up processing, but additional threads increase the memory requirement linearly.
Comments.UserDisplayName: Konrad Rudolph
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-05-24T07:52:35.363Z
Comments.Text: @KonradRudolph, why not put your comment as an answer? I would certainly upvote if it was an answer.
Comments.UserDisplayName: benn
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-05-24T09:33:19.327Z
Comments.Text: @b.nota Fair enough, upon reflection it would probably solve OP’s problem.
Comments.UserDisplayName: Konrad Rudolph
Tags: <rna-seq><rna-alignment><star>
Title: Error creating indices using STAR
ThreadText: # Error creating indices using STAR
  
  (Asked by: username_0 on 2018-05-23)
  
  I am trying to index wheat genome using STAR through following command
  
  ```
  STAR --runMode genomeGenerate --genomeFastaFiles Triticum_aestivum.TGACv1.dna_sm.toplevel.fa --runThreadN 28 
  
  ```
  
  But getting following error,
  
  ```
  terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
  Aborted (core dumped)
  
  ```
  
  The size of the genome is 13.5 GB and the server capacity is 32 cores with 125 GB RAM, how this error can be resolved?
  
  Memory usage: [![enter image description here](https://i.sstatic.net/uKcEQ.png)](https://i.sstatic.net/uKcEQ.png)
  
  I have decreased the number of threads to 8 but again getting the same error as shown below:
  
  ```
  > STAR   --runMode genomeGenerate   --runThreadN 8   --genomeDir wheat   --genomeFastaFiles wheat.fa                                                                      
  Jun 04 02:13:31 ..... started STAR run                                                                                                                                                                                           
  Jun 04 02:13:31 ... starting to generate Genome files                                                                                                                                                                            
  terminate called after throwing an instance of 'std::bad_alloc'                                                                                                                                                                  
  what():  std::bad_alloc                                                                                                                                                                                                        
  Aborted (core dumped)                                           
  
  ```
  
  Even I have tried to use STAR without any threads but still the same error:
  
  ```
  > STAR   --runMode genomeGenerate   --genomeDir wheat   --genomeFastaFiles wheat.fa                                                                         
  Jun 04 02:19:44 ..... started STAR run                                                                                                                                                                                           
  Jun 04 02:19:44 ... starting to generate Genome files                                                                                                                                                                            
  terminate called after throwing an instance of 'std::bad_alloc'                                                                                                                                                                  
  what():  std::bad_alloc
  Aborted (core dumped)
  
  ```
  
  May there can be an issue in reference of wheat that I am using, I got this reference from [here.](ftp://ftp.ensemblgenomes.org/pub/plants/release-39/fasta/triticum_aestivum/dna/Triticum_aestivum.TGACv1.dna.toplevel.fa.gz)
  Here are the first ten lines of the reference using head command:
  
  ```
  > head wheat.fa
  >Pt dna:chromosome chromosome:TGACv1:Pt:1:135835:1 REF                                                                                                                                                                           
  ACAGAAATACCCAATATCTTGTTCTAGCAAGATATTGGGTATTTCTGTCTTTTCTTTCTT                                                                                                                                                                     
  CAAAAATTCTTATATGTTAGCAGAAAAACCTTATCCATTAATAGATGGAACTTCAACAGC                                                                                                                                                                     
  AGCTAAGTCTAGAGGGAAGTTGTGAGCATTACGTTCGTGCATTACTTCCATACCAAGGTT                                                                                                                                                                     
  AGCACGGTTGATGATATCAGCCCAAGTATTAATAACGCGACCTTGACTATCAACTACAGA                                                                                                                                                                     
  TTGGTTGAAATTGAAACCATTTAGGTTGAAAGCCATAGTACTAATACCTAAAGCAGTGAA                                                                                                                                                                     
  CCAGATTCCTACTACAGGCCAAGCAGCCAAGAAGAAGTGTAAAGAACGAGAGTTGTTGAA                                                                                                                                                                     
  ACTAGCATATTGGAAGATTAATCGGCCAAAATAACCATGAGCAGCCACAATATTATAAGT                                                                                                                                                                     
  TTCTTCCTCTTGACCAAATTTGTAACCCTCATTAGCAGATTCATTTTCAGTAGTTTCCCT                                                                                                                                                                     
  GATCAAACTAGAGGTTACCAAGGAACCATGCATAGCACTGAATAGGGAACCGCCGAATAC 
  
  ```
  
  How this problem can be resolved?
  
  **Tags:** rna-seq, rna-alignment, star
  
  > **Comments:**
  > **username_1**: Keep an eye on the memory usage and see if you're actually exceeding what's available.  
  > **username_0**: The memory does not exceeds 1.6 usage that is normal,  
  > **username_0**: Here is the [link](https://imgur.com/gallery/lRyo3vW) to the memory usage  
  > **username_1**: Try removing the `--sjdbGTF...` stuff from your command as it's likely there's an issue either with the GFF3 file or the fasta file.  
  > **username_2**: Use (much!) fewer threads. Threads speed up processing, but additional threads increase the memory requirement linearly.  
  > **username_3**: @username_2, why not put your comment as an answer? I would certainly upvote if it was an answer.  
  > **username_2**: @b.nota Fair enough, upon reflection it would probably solve OP’s problem.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2018-06-04. Score: 3 | Accepted answer
  
  In addition to the other answer (to reduce the number of threads), you might also need to add a `--genomeChrBinNbits` argument. Your reference has more than 5000 scaffolds, so you need to adjust the `--genomeChrBinNbits` argument to reduce RAM consumption. Found [here](https://github.com/alexdobin/STAR/issues/103).
  
  They recommend:
  
  > --genomeChrBinNbits} = min(18, log2(GenomeLength/NumberOfReferences))
  
  In your case:
  
  Genome size 17G, number of scaffolds 735945
  
  So `--genomeChrBinNbits` should be set to log2(17,000,000,000/735,945) = 14.4955, so let's say 14 or 15.
  
  > **Comments:**
  > **username_0**: After your command it gave error to increase ram using limitGenomeGenerateRAM, then I increased the ram using limitGenomeGenerateRAM and finally it worked fine, thanks.  
  > **username_3**: @username_0, I am glad it finally works! Thanks for your feedback.  
  
  ---
  
  ### Answer by username_2 on 2018-05-24. Score: 5
  
  Try reducing the number of threads.
  
  When multithreading the index creation (and other memory bound tasks), the memory usage increases linearly with the number of threads. If you required a 10 GiB working space work with a single thread, then the requirement for 10 threads would be at 100 GiB.
  
  STAR is particularly memory hungry for the index creation to start with, and your genome is exceptionally large. I’ve had STAR require > 32 GiB of memory for the human genome, with a moderate number of threads. I’m not surprised that it fails with a 4× larger genome and many more threads: it would probably require at least 2× as much RAM as your machine has.
  
  > **Comments:**
  > **username_0**: Thanks for your answer but it didn't work.  
  > **username_0**: I have again edited my post to add more details to my problem.  
  
  ---
  
  