Answers: <p>The problem can easily be solved using the following components:</p>
  
  <ul>
  <li><p>On the instances serving your <em>webapp</em> continue to monitor the number of incoming requests – and anything else you see fit.</p></li>
  <li><p>Publish the number of incoming requests to a monitoring system. If it is not yet implemented, this step will improve your monitoring capacities, and will help you to monitor the load on each hosts as well as host balancing.</p></li>
  <li><p>From the incoming requests, deduce an estimated required number of <em>webaspp</em> servers needed to serve that work load, as well as the difference between the actual number and the estimated needed number. In the case you describe, it seems that the estimated number is just a scalar function of the total number of incoming requests on a recent period of time. On other systems or after some times, more subtle strategies can be implemented. Monitoring these quantities and the difference could ease the traceability of the auto-scaling strategy, and will monitor its responsivity.</p></li>
  <li><p>Last implement the auto-scaling itself, at this point, this is really just reading some number from your monitoring system and writing it to your scaling system.</p></li>
  </ul>
  
Answers: CC BY-SA 3.0
Answers: 2017-04-20T08:17:56.197Z
Answers: 2017-04-20T08:17:56.197Z
Answers: Michaël Le Barbier
Answers: <p>One possible approach is to allow such instances to make demands for new instances based on local request counters, but instead of directly reacting to those demands you would funnel them to a central instance creation logic.</p>
  
  <p>That logic would immediately react to the first demand, but also start a "cool off" countdown timer. Any subsequent demand received while the timer is still active would be considered to be caused by the same traffic spike that triggered the first demand and would thus be ignored.</p>
  
  <p>A similar logic could be used to gradually shutdown idle instances, if the local counters remain below a minimum level.</p>
  
  <p><strong>Note:</strong> the local counters would need to operate in a leaky bucket manner or be periodically reset for such approach to be possible - so that the demands are repeated if the traffic remains high.</p>
  
  <p>Another possible approach is to just publish the local counters. A central piece of logic would periodically collect and aggregate these local counter values in order to decide on lauching new instances or shutting down existing ones. </p>
  
  <p>The advantage of such method is the lack of a <em>single</em> global counter which would require write access locking (to prevent corruption) which would be a scalability limitation. </p>
  
  <p>This method is described in mode details in <a href="https://cloud.google.com/appengine/articles/sharding_counters" rel="nofollow noreferrer">Sharding counters</a>.</p>
  
Answers: CC BY-SA 3.0
Answers: 2017-04-20T18:49:18.687Z
Answers: 2017-04-20T18:49:18.687Z
Answers: Dan Cornilescu
Body: <p>Somewhere around I got to know about different approaches which were used to scale our webapp which included <strong>Scaling using local request counters</strong>. Below that they had written the drawbacks of this approach adding that </p>
  
  <blockquote>
    <p>Each instance would reach the threshold almost at the same time and
    hence each would demand a new instance, leading to a large number of
    instances even though the number should have increased by one only</p>
  </blockquote>
  
  <p>I was curious to know if there's a solution to this problem or any workaround ?</p>
  
Comments: CC BY-SA 3.0
Comments: 2017-04-10T11:56:21.760Z
Comments: Yes don't use local counters, use a central system to handle the overall load and scale accordingly. I don't get what more you are after, your quote sounds absolutely clear for me. Could you elaborate on what you are missing ?
Comments: Tensibai
Comments: CC BY-SA 3.0
Comments: 2017-04-10T13:56:14.447Z
Comments: What if you named the instance you are creating the number that is being requested, and thus when you go to try to create "instance_5" the name is already in use and can't be created?
Comments: avi
Tags: <containers><autoscaling>
Title: Autoscaling containers using local request counters
UserDisplayName: radbrawler
ThreadText: # Autoscaling containers using local request counters
  
  (Asked by: username_0 on 2017-04-10)
  
  Somewhere around I got to know about different approaches which were used to scale our webapp which included **Scaling using local request counters**. Below that they had written the drawbacks of this approach adding that
  
  > Each instance would reach the threshold almost at the same time and
  > hence each would demand a new instance, leading to a large number of
  > instances even though the number should have increased by one only
  
  I was curious to know if there's a solution to this problem or any workaround ?
  
  **Tags:** containers, autoscaling
  
  > **Comments:**
  > **username_1**: Yes don't use local counters, use a central system to handle the overall load and scale accordingly. I don't get what more you are after, your quote sounds absolutely clear for me. Could you elaborate on what you are missing ?  
  > **username_2**: What if you named the instance you are creating the number that is being requested, and thus when you go to try to create "instance_5" the name is already in use and can't be created?  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2017-04-20. Score: 3
  
  The problem can easily be solved using the following components:
  
  * On the instances serving your *webapp* continue to monitor the number of incoming requests – and anything else you see fit.
  * Publish the number of incoming requests to a monitoring system. If it is not yet implemented, this step will improve your monitoring capacities, and will help you to monitor the load on each hosts as well as host balancing.
  * From the incoming requests, deduce an estimated required number of *webaspp* servers needed to serve that work load, as well as the difference between the actual number and the estimated needed number. In the case you describe, it seems that the estimated number is just a scalar function of the total number of incoming requests on a recent period of time. On other systems or after some times, more subtle strategies can be implemented. Monitoring these quantities and the difference could ease the traceability of the auto-scaling strategy, and will monitor its responsivity.
  * Last implement the auto-scaling itself, at this point, this is really just reading some number from your monitoring system and writing it to your scaling system.
  
  
  ---
  
  ### Answer by username_4 on 2017-04-20. Score: 2
  
  One possible approach is to allow such instances to make demands for new instances based on local request counters, but instead of directly reacting to those demands you would funnel them to a central instance creation logic.
  
  That logic would immediately react to the first demand, but also start a "cool off" countdown timer. Any subsequent demand received while the timer is still active would be considered to be caused by the same traffic spike that triggered the first demand and would thus be ignored.
  
  A similar logic could be used to gradually shutdown idle instances, if the local counters remain below a minimum level.
  
  **Note:** the local counters would need to operate in a leaky bucket manner or be periodically reset for such approach to be possible - so that the demands are repeated if the traffic remains high.
  
  Another possible approach is to just publish the local counters. A central piece of logic would periodically collect and aggregate these local counter values in order to decide on lauching new instances or shutting down existing ones.
  
  The advantage of such method is the lack of a *single* global counter which would require write access locking (to prevent corruption) which would be a scalability limitation.
  
  This method is described in mode details in [Sharding counters](https://cloud.google.com/appengine/articles/sharding_counters).
  
  
  ---
  
  