Answers.Body: <p>That <code>Killed</code> message is what you see when a process is killed using a <a href="https://www.fosslinux.com/121761/the-abcs-of-linux-signals-sigint-sigterm-and-sigkill-explained.htm" rel="nofollow noreferrer">KILL signal</a>:</p>
  <p><a href="https://i.sstatic.net/yDzcmO0w.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/yDzcmO0w.png" alt="screenshot of a terminal showing that when a process receives SIGKILL, it dies and prints out Killed." /></a></p>
  <p>This is the signal that the <a href="https://docs.rackspace.com/docs/linux-out-of-memory-killer" rel="nofollow noreferrer">out of memory (OOM) killer</a>, the kernel process whose job it is to stop processes if they are using too much memory. Instead of allowing the system to hang, the kernel starts killing things that are hogging the memory.</p>
  <p>I have never used <code>pbmm2</code> myself, but I checked <a href="https://www.pacb.com/wp-content/uploads/SMRT-Tools-Reference-Guide-v8.0.pdf" rel="nofollow noreferrer">the documentation</a> and it doesn't appear to have any options for memory management. This absence suggests that it doesn't really expect memory to be an issue, in which case it is surprising that you got that error.</p>
  <p>Could it be that you are launching this on the host node of your cluster and the cluster is designed to work with a queuing system that sends jobs to the full cluster instead? Or perhaps the system was under very heavy use and other users' jobs had taken all the RAM leaving none for you? Essentially, it looks like you will need to run this on a machine with more RAM.</p>
  <p>Alternatively, on the off chance that the tool is just badly designed and doesn't manage memory well, since all you want to do is make an index, you could try using <a href="https://github.com/lh3/minimap2?tab=readme-ov-file#full-genome" rel="nofollow noreferrer"><code>minimap2</code></a> in case that works better. You <a href="https://bioinformatics.stackexchange.com/a/11566/298">can also not create an index</a> and let <code>pbmm2</code> create it on the fly, assuming it works like <code>minimap2</code>. However, if your system doesn't have enough memory to make the index, I really doubt it will have enough to create the bam either, bringing us back to: use a machine with more RAM.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-06-13T08:02:30.380Z
Answers.LastActivityDate: 2024-06-13T08:02:30.380Z
Answers.UserDisplayName: terdon
Answers.Body: <p>I got the same problem on an 8GB machine, where there was only about 4GB memory left available. I was able to build an index on a bigger machine with more memory.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-10T15:39:54.053Z
Answers.LastActivityDate: 2024-10-10T15:39:54.053Z
Answers.UserDisplayName: SmallChess
Body: <p>I am trying to use pbmm2 to create a reference genome index.
  I'm on a cluster server, I don't know if that's the reason why it has been killed.</p>
  <pre><code>pbmm2 index hg38.fa hg38.mmi
  </code></pre>
  <p><a href="https://i.sstatic.net/TpM8CMOJ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/TpM8CMOJ.png" alt="enter image description here" /></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-12T19:17:08.123Z
Comments.Text: You're probably running out of memory. Please Google before asking, this is a Linux question and not specific to bioinformatics.
Comments.UserDisplayName: Ram RS
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-12T20:06:31.713Z
Comments.Text: @RamRS I have tried with ulimit -m or -v ~64GB; I don't know how many GBs would be needed for a human genome, previously it was indicated unlimited.
Comments.UserDisplayName: María José
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-12T20:39:38.640Z
Comments.Text: How many gigs depends on the program. How much physical memory do you have access to though? Given that pbmm2 is essentially minimap2 and minimap2 was developed by Heng Li, I'd say it should be pretty efficient, but try using minimap2's indexing operation instead of pbmm2's.
Comments.UserDisplayName: Ram RS
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-13T07:59:48.887Z
Comments.Text: Are you running this job on the head node? It has limited resources, and is only supposed to be used to communicate jobs to the compute nodes. Seek out your server administrator and see if they have a wiki or other instructions on how to run jobs on their cluster.
Comments.UserDisplayName: Pallie
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-16T08:32:22.500Z
Comments.Text: @Pallie assuming that the cluster is running SLURM the queues/partition looking at `/etc/slurm/slurm.conf`
Comments.UserDisplayName: darked89
Tags: <long-reads><pacbio><indexing><genome-index>
Title: Indexing the reference genome my process is killed
ThreadText: # Indexing the reference genome my process is killed
  
  (Asked by: username_0 on 2024-06-12)
  
  I am trying to use pbmm2 to create a reference genome index.
  I'm on a cluster server, I don't know if that's the reason why it has been killed.
  
  ```
  pbmm2 index hg38.fa hg38.mmi
  
  ```
  
  [![enter image description here](https://i.sstatic.net/TpM8CMOJ.png)](https://i.sstatic.net/TpM8CMOJ.png)
  
  **Tags:** long-reads, pacbio, indexing, genome-index
  
  > **Comments:**
  > **username_1**: You're probably running out of memory. Please Google before asking, this is a Linux question and not specific to bioinformatics.  
  > **username_0**: @username_1 I have tried with ulimit -m or -v ~64GB; I don't know how many GBs would be needed for a human genome, previously it was indicated unlimited.  
  > **username_1**: How many gigs depends on the program. How much physical memory do you have access to though? Given that pbmm2 is essentially minimap2 and minimap2 was developed by Heng Li, I'd say it should be pretty efficient, but try using minimap2's indexing operation instead of pbmm2's.  
  > **username_2**: Are you running this job on the head node? It has limited resources, and is only supposed to be used to communicate jobs to the compute nodes. Seek out your server administrator and see if they have a wiki or other instructions on how to run jobs on their cluster.  
  > **username_3**: @username_2 assuming that the cluster is running SLURM the queues/partition looking at `/etc/slurm/slurm.conf`  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_4 on 2024-06-13. Score: 1
  
  That `Killed` message is what you see when a process is killed using a [KILL signal](https://www.fosslinux.com/121761/the-abcs-of-linux-signals-sigint-sigterm-and-sigkill-explained.htm):
  
  [![screenshot of a terminal showing that when a process receives SIGKILL, it dies and prints out Killed.](https://i.sstatic.net/yDzcmO0w.png)](https://i.sstatic.net/yDzcmO0w.png)
  
  This is the signal that the [out of memory (OOM) killer](https://docs.rackspace.com/docs/linux-out-of-memory-killer), the kernel process whose job it is to stop processes if they are using too much memory. Instead of allowing the system to hang, the kernel starts killing things that are hogging the memory.
  
  I have never used `pbmm2` myself, but I checked [the documentation](https://www.pacb.com/wp-content/uploads/SMRT-Tools-Reference-Guide-v8.0.pdf) and it doesn't appear to have any options for memory management. This absence suggests that it doesn't really expect memory to be an issue, in which case it is surprising that you got that error.
  
  Could it be that you are launching this on the host node of your cluster and the cluster is designed to work with a queuing system that sends jobs to the full cluster instead? Or perhaps the system was under very heavy use and other users' jobs had taken all the RAM leaving none for you? Essentially, it looks like you will need to run this on a machine with more RAM.
  
  Alternatively, on the off chance that the tool is just badly designed and doesn't manage memory well, since all you want to do is make an index, you could try using [`minimap2`](https://github.com/lh3/minimap2?tab=readme-ov-file#full-genome) in case that works better. You [can also not create an index](https://bioinformatics.stackexchange.com/a/11566/298) and let `pbmm2` create it on the fly, assuming it works like `minimap2`. However, if your system doesn't have enough memory to make the index, I really doubt it will have enough to create the bam either, bringing us back to: use a machine with more RAM.
  
  
  ---
  
  ### Answer by username_5 on 2024-10-10. Score: 0
  
  I got the same problem on an 8GB machine, where there was only about 4GB memory left available. I was able to build an index on a bigger machine with more memory.
  
  
  ---
  
  