Answers.Body: <p>You need to diagnose for low memory and consider switching to an <strong>R5</strong> processor that prioritises memory. Basically it's easily solvable by switching hardware. There are certainly alot of other approaches but this is specialist HPC.</p>
  <p>Personally, I would use Slurm ParallelCluster in AWS. This is very easy to set up from from the configuration file.</p>
  <p>The overview of instance types here.
  <a href="https://aws.amazon.com/ec2/instance-types/" rel="nofollow noreferrer">https://aws.amazon.com/ec2/instance-types/</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-14T14:23:47.307Z
Answers.LastActivityDate: 2024-02-14T14:23:47.307Z
Answers.UserDisplayName: M__
Body: <p>I have 16 CPUs on my AWS EC2 instance and I am trying to use <a href="https://sourceforge.net/p/ksnp/discussion/general/" rel="nofollow noreferrer">kSNP4</a> to generate an SNP matrix. I was able to successfully run this with over 150 samples - while it took a while to run it did complete - and now I want to run it with over 170 samples but when I use the same command it dies at the second step of the process. I have reason to believe it is due to the processors on my server that I am using more than the tool itself.</p>
  <p>Does anyone know the best way to ensure maximum processing ability to run tools on an EC2 instance? Or what I can do to increase the power of the instance?</p>
  <p>I know this is more than a technical question than a specific bioinformatic question but I would appreciate any insight!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-12T17:32:43.683Z
Comments.Text: Do you want to increase CPUs or memory? They are very different things. In either case though, this isn't really something we can help with and you should be talking to your sysadmins or AWS help. Sounds like you need a machine with different specs.
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-12T20:36:27.737Z
Comments.Text: AWS docs outlines how to pick a different instance type for scaling: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-resize.html I don't see why a tool that can process 150 samples will crash with 170 unless the complexity goes up geometrically or exponentially somehow
Comments.UserDisplayName: Ram RS
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-16T17:45:23.053Z
Comments.Text: To be honest ... it looks like a bug in the data. 150 -> 170 is not huge and "crashing" is ominous.
Comments.UserDisplayName: M__
Tags: <phylogenetics><phylogeny><linux><aws>
Title: Increasing memory usage in AWS instance
ThreadText: # Increasing memory usage in AWS instance
  
  (Asked by: username_0 on 2024-02-12)
  
  I have 16 CPUs on my AWS EC2 instance and I am trying to use [kSNP4](https://sourceforge.net/p/ksnp/discussion/general/) to generate an SNP matrix. I was able to successfully run this with over 150 samples - while it took a while to run it did complete - and now I want to run it with over 170 samples but when I use the same command it dies at the second step of the process. I have reason to believe it is due to the processors on my server that I am using more than the tool itself.
  
  Does anyone know the best way to ensure maximum processing ability to run tools on an EC2 instance? Or what I can do to increase the power of the instance?
  
  I know this is more than a technical question than a specific bioinformatic question but I would appreciate any insight!
  
  **Tags:** phylogenetics, phylogeny, linux, aws
  
  > **Comments:**
  > **username_1**: Do you want to increase CPUs or memory? They are very different things. In either case though, this isn't really something we can help with and you should be talking to your sysadmins or AWS help. Sounds like you need a machine with different specs.  
  > **username_2**: AWS docs outlines how to pick a different instance type for scaling: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-resize.html I don't see why a tool that can process 150 samples will crash with 170 unless the complexity goes up geometrically or exponentially somehow  
  > **username_3**: To be honest ... it looks like a bug in the data. 150 -> 170 is not huge and "crashing" is ominous.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2024-02-14. Score: 1
  
  You need to diagnose for low memory and consider switching to an **R5** processor that prioritises memory. Basically it's easily solvable by switching hardware. There are certainly alot of other approaches but this is specialist HPC.
  
  Personally, I would use Slurm ParallelCluster in AWS. This is very easy to set up from from the configuration file.
  
  The overview of instance types here.
  <https://aws.amazon.com/ec2/instance-types/>
  
  
  ---
  
  