Answers.Body: <p>So, I &quot;fixed&quot; the problem by building with gcc 4.9.4, rather than gcc 9.2.0.</p>
  <p>Why it's necessary to build with that much earlier, and completely out of date, version, I don't know.</p>
  <p>But it worked</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-08T22:46:28.383Z
Answers.LastActivityDate: 2024-05-08T22:46:28.383Z
Answers.UserDisplayName: Greg Dougherty
Body: <p>Also posted <a href="https://www.biostars.org/p/9591903/" rel="nofollow noreferrer">on biostars</a></p>
  <p>I'm trying to run <a href="https://github.com/Illumina/manta" rel="nofollow noreferrer">manta</a> on RHEL 8.  I built it using gcc 9.2.0 with --build-type=RelWithDebInfo</p>
  <p>The following command throws a &quot;segmentation fault, core dumped&quot;.  Running it under gdb, doing a &quot;bt&quot; when it crashes, consistently gets me a stack trace with 8733 frames, of which ~8700 are</p>
  <pre><code>searchRepeats (opt=..., index=aNumber, word=..., wordIndices=..., wordStack=..., repeatWords=...) at manta-1.6.0.release_src/src/c++/lib/assembly/IterativeAssembler.cpp:586
  </code></pre>
  <p>Yes, always that line.  aNumber ranges from 20 up to 33319.</p>
  <pre><code>manta/1.6.0/libexec/GenerateSVCandidates \
    --threads 10 --align-stats alignmentStats.xml\
     --graph-file svLocusGraph.bin --bin-index 0 --bin-count 1 \
    --max-edge-count 10 --min-candidate-sv-size 8 \
    --min-candidate-spanning-count 3 --min-scored-sv-size 50 \
    --ref GCA_000001405.15_GRCh38_no_alt_short_headers_nonACTG_to_N.fa \
    --candidate-output-file svHyGen/candidateSV.0000.vcf \
    --tumor-output-file svHyGen/tumorSV.0000.vcf \
    --chrom-depth chromDepth.txt --edge-runtime-log svHyGen/edgeRuntimeLog.0000.txt \
    --edge-stats-log results/stats/svCandidateGenerationStats.xml \
    --edge-stats-report results/stats/svCandidateGenerationStats.tsv \
    --tumor-align-file sample.bam
  </code></pre>
  <p>These are the top stack frames.  So I'm assuming the problem is running out of memory</p>
  <pre><code>#0  0x0000155554637b4e in _int_malloc () from /lib64/libc.so.6
  No symbol table info available.
  #1  0x0000155554639982 in malloc () from /lib64/libc.so.6
  No symbol table info available.
  #2  0x000000000070ddd5 in operator new (sz=81) at /research/bsi/tools/shared/gcc-9.2.0/libstdc++-v3/libsupc++/new_op.cc:50
          p = &lt;optimized out&gt;
  #3  0x000000000078920d in std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;::_M_mutate (this=this@entry=0x15555419d1f0, __pos=__pos@entry=40, __len1=&lt;optimized out&gt;, __s=__s@entry=0x0, __len2=__len2@entry=1)
      at /research/bsi/tools/shared/build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.tcc:310
          __how_much = 0
          __new_capacity = 80
          __r = &lt;optimized out&gt;
  </code></pre>
  <ol>
  <li><p>This is a loaded system.  How do I give manta more memory?</p>
  </li>
  <li><p>Has anyone seen this problem?</p>
  </li>
  <li><p>The bam file is only 23 gb</p>
  </li>
  <li><p>I ran it under  /usr/bin/time -v.  I got</p>
  <p>Maximum resident set size (kbytes): 1084964</p>
  </li>
  </ol>
  <p>I've got a lot more than 1gb of ram available.  How do I get the manta tool to use it?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-04-05T11:07:58.773Z
Comments.Text: What is a loaded system? How are you checking that you have more than 1G RAM? Is it that it's running out of RAM (and, if so, do you have swap enabled?) or that a specific variable in the code is trying to store more data than the amount of memory allocated for that variable? Have you tried asking the manta devs?
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-04-05T11:09:58.390Z
Comments.Text: https://github.com/Illumina/manta/blob/master/docs/userGuide/README.md#runtime-hardware-requirements mentions "<1GB _per core_" (emphasis mine). Please [edit] and tell us about your hardware.
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-04-08T16:53:20.907Z
Comments.Text: @terdon Running on the Cluster I gave it 10 cores and 80GB.  Debugging is on a node that has 64 cores and a terabyte of memory.  It's doing a seg fault during a "new" call, so it's running out of "available to it" memory, but it's coming nowhere near to exhausting the memory available on the system
Comments.UserDisplayName: Greg Dougherty
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-04-23T00:50:18.003Z
Comments.Text: Just following up; did you work out the cause and/or a solution this problem @GregDougherty?
Comments.UserDisplayName: jared_mamrot
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-05-08T22:47:06.210Z
Comments.Text: @jared_mamrot I built it with a much earlier version of gcc, and that fixed the problem
Comments.UserDisplayName: Greg Dougherty
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-05-09T00:02:54.060Z
Comments.Text: That's surprising! I tried to replicate the issue in a CentOS VM but failed, so I'm glad you figured out a solution :)
Comments.UserDisplayName: jared_mamrot
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-05-28T19:34:16.040Z
Comments.Text: @jared_mamrot thank you.  I think it only happened to us once we got on RHEL 8
Comments.UserDisplayName: Greg Dougherty
Tags: <untagged>
Title: Debugging manta 1.6.0 seg fault
ThreadText: # Debugging manta 1.6.0 seg fault
  
  (Asked by: username_0 on 2024-04-05)
  
  Also posted [on biostars](https://www.biostars.org/p/9591903/)
  
  I'm trying to run [manta](https://github.com/Illumina/manta) on RHEL 8. I built it using gcc 9.2.0 with --build-type=RelWithDebInfo
  
  The following command throws a "segmentation fault, core dumped". Running it under gdb, doing a "bt" when it crashes, consistently gets me a stack trace with 8733 frames, of which ~8700 are
  
  ```
  searchRepeats (opt=..., index=aNumber, word=..., wordIndices=..., wordStack=..., repeatWords=...) at manta-1.6.0.release_src/src/c++/lib/assembly/IterativeAssembler.cpp:586
  
  ```
  
  Yes, always that line. aNumber ranges from 20 up to 33319.
  
  ```
  manta/1.6.0/libexec/GenerateSVCandidates \
    --threads 10 --align-stats alignmentStats.xml\
     --graph-file svLocusGraph.bin --bin-index 0 --bin-count 1 \
    --max-edge-count 10 --min-candidate-sv-size 8 \
    --min-candidate-spanning-count 3 --min-scored-sv-size 50 \
    --ref GCA_000001405.15_GRCh38_no_alt_short_headers_nonACTG_to_N.fa \
    --candidate-output-file svHyGen/candidateSV.0000.vcf \
    --tumor-output-file svHyGen/tumorSV.0000.vcf \
    --chrom-depth chromDepth.txt --edge-runtime-log svHyGen/edgeRuntimeLog.0000.txt \
    --edge-stats-log results/stats/svCandidateGenerationStats.xml \
    --edge-stats-report results/stats/svCandidateGenerationStats.tsv \
    --tumor-align-file sample.bam
  
  ```
  
  These are the top stack frames. So I'm assuming the problem is running out of memory
  
  ```
  #0  0x0000155554637b4e in _int_malloc () from /lib64/libc.so.6
  No symbol table info available.
  #1  0x0000155554639982 in malloc () from /lib64/libc.so.6
  No symbol table info available.
  #2  0x000000000070ddd5 in operator new (sz=81) at /research/bsi/tools/shared/gcc-9.2.0/libstdc++-v3/libsupc++/new_op.cc:50
          p = <optimized out>
  #3  0x000000000078920d in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate (this=this@entry=0x15555419d1f0, __pos=__pos@entry=40, __len1=<optimized out>, __s=__s@entry=0x0, __len2=__len2@entry=1)
      at /research/bsi/tools/shared/build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.tcc:310
          __how_much = 0
          __new_capacity = 80
          __r = <optimized out>
  
  ```
  
  1. This is a loaded system. How do I give manta more memory?
  2. Has anyone seen this problem?
  3. The bam file is only 23 gb
  4. I ran it under /usr/bin/time -v. I got
  
     Maximum resident set size (kbytes): 1084964
  
  I've got a lot more than 1gb of ram available. How do I get the manta tool to use it?
  
  **Tags:** untagged
  
  > **Comments:**
  > **username_1**: What is a loaded system? How are you checking that you have more than 1G RAM? Is it that it's running out of RAM (and, if so, do you have swap enabled?) or that a specific variable in the code is trying to store more data than the amount of memory allocated for that variable? Have you tried asking the manta devs?  
  > **username_1**: https://github.com/Illumina/manta/blob/master/docs/userGuide/README.md#runtime-hardware-requirements mentions "<1GB _per core_" (emphasis mine). Please [edit] and tell us about your hardware.  
  > **username_0**: @username_1 Running on the Cluster I gave it 10 cores and 80GB.  Debugging is on a node that has 64 cores and a terabyte of memory.  It's doing a seg fault during a "new" call, so it's running out of "available to it" memory, but it's coming nowhere near to exhausting the memory available on the system  
  > **username_2**: Just following up; did you work out the cause and/or a solution this problem @username_0?  
  > **username_0**: @username_2 I built it with a much earlier version of gcc, and that fixed the problem  
  > **username_2**: That's surprising! I tried to replicate the issue in a CentOS VM but failed, so I'm glad you figured out a solution :)  
  > **username_0**: @username_2 thank you.  I think it only happened to us once we got on RHEL 8  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2024-05-08. Score: 1 | Accepted answer
  
  So, I "fixed" the problem by building with gcc 4.9.4, rather than gcc 9.2.0.
  
  Why it's necessary to build with that much earlier, and completely out of date, version, I don't know.
  
  But it worked
  
  
  ---
  
  