Answers: <p><em>(N.B. this answer is, in the spirit of the question, high-level and not very concrete...)</em></p>
  
  <blockquote>
    <p>How can I calculate throughput of our system</p>
  </blockquote>
  
  <p>If by "calculate" you mean to take some data points, dry run some formulas and get a pretty close approximation to your throughput - that is pretty much impossible, unless you have a <em>lot</em> more information - which may also include pretty random parts, hence quite hard to do theoretically.</p>
  
  <p>If you mean "measure", then it could be as simple as checking your logfiles and counting lines per time unit. </p>
  
  <blockquote>
    <p>and build monitoring around max load that our system can handle?</p>
  </blockquote>
  
  <p>Unless you're NASA, that's probably a case for trial and error. Run your system(s) and see how much troughput you get. Increase worker nodes. See what happens.</p>
  
  <p>If you already know the behaviour of your overal system - i.e., whether it scales linearly, whether there are bottlenecks like databases, lock contention and such, then you can take shortcuts or good guesses.</p>
  
  <p>That's one of the reasons for doing the container-based scaling we do these days - you can throw a few more workers at it relatively easily. </p>
  
  <p>How you actually (technically) do that depends on what platform you are using. AWS, Kubernetes/OpenShift etc. come with techniques or settings do do it automatically for you. I assume those work mainly with the metric of "free workers" - trying to get the "free:busy worker" ratio into some target corridor so that every new request has a very high likelihood to hit a free worker, at any time, thus getting (theoretical) constant time for each request.</p>
  
Answers: CC BY-SA 4.0
Answers: 2018-09-25T20:18:09.593Z
Answers: 2018-09-25T20:18:09.593Z
Answers: AnoE
Answers: <p>KPI (key performance indicator) or software metrics could help. For starters after a bit googling, Kubernetes has self-healing mechanism in which whether thresholds could be set up to heal after exceeding some measures is something I never tried. I'd love to know if it is doable though.</p>
  
  <p>Read: <a href="https://github.com/DataDog/the-monitor/blob/master/kubernetes/monitoring-kubernetes-performance-metrics.md" rel="nofollow noreferrer">Monitoring Kubernetes performance metrics</a>.</p>
  
Answers: CC BY-SA 4.0
Answers: 2018-08-25T10:12:33.190Z
Answers: 2018-11-06T12:12:52.043Z
Answers: 2018-11-06T12:12:52.043Z
Answers: hakkican
Body: <p>I am trying to answer a question:</p>
  <h3>Do we need more worker machines to handle current load?</h3>
  <p>We have a variable number of jobs coming in and each job takes different time. Here is a snapshot of our system for a period 12 hours.</p>
  <p><a href="https://i.sstatic.net/tniGY.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/tniGY.png" alt="Jobs in 8 hours" /></a></p>
  <p>But the number of jobs in system changes based on day and time.</p>
  <p>How can I calculate throughput of our system and build monitoring around max load that our system can handle?</p>
  
Comments: CC BY-SA 4.0
Comments: 2018-08-24T14:53:51.327Z
Comments: You're looking at math and queuing theory. Your variables will likely be median job-size, time to process that median job, and a number derived on how many simultaneous jobs your server can handle. That's a bit beyond me at the moment.
Comments: sysadmin1138
Comments: CC BY-SA 4.0
Comments: 2018-08-24T16:56:29.540Z
Comments: You may be asking the wrong question. What's your SLA or target for queue processing?
Comments: CodeGnome
Comments: CC BY-SA 4.0
Comments: 2018-08-24T18:16:39.300Z
Comments: Actually I am defining SLA and this exercise is part of coming up with SLA metrics.
Comments: Rohit
Comments: CC BY-SA 4.0
Comments: 2018-08-25T17:38:08.013Z
Comments: Your question lacks some information, the graph provided means nothing related to worker machines except numbers jobs in bucket of duration. Server capacity has to be determined on one or several servers resources (disks throughput, disk storage space, CPU usage, memory usage) based on what jobs relies on, and compare these resource usage with the capacity you can process the current data. You need to cross these two information over the time to see how your current worker capacity can process the workload in 3, 6, 12 months. I over-simplified naturally because of comments length limitation.
Comments: Baptiste Mille-Mathias
Tags: <capacity-planning>
Title: How to calculate server capacity?
UserDisplayName: Rohit
ThreadText: # How to calculate server capacity?
  
  (Asked by: username_0 on 2018-08-23)
  
  I am trying to answer a question:
  
  ### Do we need more worker machines to handle current load?
  
  We have a variable number of jobs coming in and each job takes different time. Here is a snapshot of our system for a period 12 hours.
  
  [![Jobs in 8 hours](https://i.sstatic.net/tniGY.png)](https://i.sstatic.net/tniGY.png)
  
  But the number of jobs in system changes based on day and time.
  
  How can I calculate throughput of our system and build monitoring around max load that our system can handle?
  
  **Tags:** capacity-planning
  
  > **Comments:**
  > **username_1**: You're looking at math and queuing theory. Your variables will likely be median job-size, time to process that median job, and a number derived on how many simultaneous jobs your server can handle. That's a bit beyond me at the moment.  
  > **username_2**: You may be asking the wrong question. What's your SLA or target for queue processing?  
  > **username_0**: Actually I am defining SLA and this exercise is part of coming up with SLA metrics.  
  > **username_3**: Your question lacks some information, the graph provided means nothing related to worker machines except numbers jobs in bucket of duration. Server capacity has to be determined on one or several servers resources (disks throughput, disk storage space, CPU usage, memory usage) based on what jobs relies on, and compare these resource usage with the capacity you can process the current data. You need to cross these two information over the time to see how your current worker capacity can process the workload in 3, 6, 12 months. I over-simplified naturally because of comments length limitation.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_4 on 2018-09-25. Score: 1
  
  *(N.B. this answer is, in the spirit of the question, high-level and not very concrete...)*
  
  > How can I calculate throughput of our system
  
  If by "calculate" you mean to take some data points, dry run some formulas and get a pretty close approximation to your throughput - that is pretty much impossible, unless you have a *lot* more information - which may also include pretty random parts, hence quite hard to do theoretically.
  
  If you mean "measure", then it could be as simple as checking your logfiles and counting lines per time unit.
  
  > and build monitoring around max load that our system can handle?
  
  Unless you're NASA, that's probably a case for trial and error. Run your system(s) and see how much troughput you get. Increase worker nodes. See what happens.
  
  If you already know the behaviour of your overal system - i.e., whether it scales linearly, whether there are bottlenecks like databases, lock contention and such, then you can take shortcuts or good guesses.
  
  That's one of the reasons for doing the container-based scaling we do these days - you can throw a few more workers at it relatively easily.
  
  How you actually (technically) do that depends on what platform you are using. AWS, Kubernetes/OpenShift etc. come with techniques or settings do do it automatically for you. I assume those work mainly with the metric of "free workers" - trying to get the "free:busy worker" ratio into some target corridor so that every new request has a very high likelihood to hit a free worker, at any time, thus getting (theoretical) constant time for each request.
  
  
  ---
  
  ### Answer by username_5 on 2018-08-25. Score: 0
  
  KPI (key performance indicator) or software metrics could help. For starters after a bit googling, Kubernetes has self-healing mechanism in which whether thresholds could be set up to heal after exceeding some measures is something I never tried. I'd love to know if it is doable though.
  
  Read: [Monitoring Kubernetes performance metrics](https://github.com/DataDog/the-monitor/blob/master/kubernetes/monitoring-kubernetes-performance-metrics.md).
  
  
  ---
  
  