Answers: <p>You can use Envoy Filter to apply limit rate. In the below example, you apply a limit rate to a workload based on workloadSelector labels.
  The main example values are that you have a bucket of 1000 tokens. Each connection takes &quot;one token&quot; and in the bucket we &quot;return&quot; 10 tokens per 5 second. The limit rate is enabled and applied to all connection towards selected workload through <code>local_rate_limit_enabled</code> and  <code> local_rate_limit_enforced</code>.</p>
  <p>References that helped me a lot to applied the limit rate to our project was:</p>
  <p><a href="https://learncloudnative.com/blog/2022-09-08-ratelimit-istio" rel="nofollow noreferrer">Limit Rate Example</a> and <a href="https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/local_rate_limit_filter#config-http-filters-local-rate-limit" rel="nofollow noreferrer">Documentation</a></p>
  <pre><code>
      apiVersion: networking.istio.io/v1alpha3
  kind: EnvoyFilter
  metadata:
    name: my-limit-rater
    namespace: mynamespace
  spec:
    workloadSelector:
      labels:
        app: &quot;myapp&quot;
        component: &quot;mycomponent&quot;
        instance: &quot;test&quot;
        version: &quot;1.0.0&quot;
    configPatches:
      - applyTo: HTTP_FILTER
        match:
          context: SIDECAR_INBOUND
          listener:
            filterChain:
              filter:
                name: 'envoy.filters.network.http_connection_manager'
        patch:
          operation: INSERT_BEFORE
          value:
            name: envoy.filters.http.local_ratelimit
            typed_config:
              '@type': type.googleapis.com/udpa.type.v1.TypedStruct
              type_url: type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
              value:
                stat_prefix: http_local_rate_limiter
                enable_x_ratelimit_headers: DRAFT_VERSION_03
                token_bucket:
                  max_tokens: 1000
                  tokens_per_fill: 10
                  fill_interval: 5
                filter_enabled:
                  runtime_key: local_rate_limit_enabled
                  default_value:
                    numerator: 100
                    denominator: HUNDRED
                filter_enforced:
                  runtime_key: local_rate_limit_enforced
                  default_value:
                    numerator: 100
                    denominator: HUNDRED
  
  </code></pre>
  
Answers: CC BY-SA 4.0
Answers: 2023-07-03T09:26:42.570Z
Answers: 2023-07-03T09:26:42.570Z
Answers: igiannak
Body: <p>I have a load testing application (jMeter) that is calling my service that is hosted in Kubernetes.</p>
  <h2>Summary</h2>
  <p>I have a service hosted in Kubernetes that seems to have limits placed on how many calls it can take.  This service uses Istio (both with a Side Car and using the Istio Ingress Gateway).
  But I cannot seem to figure out what is limiting the calls to it.</p>
  <h2>Details</h2>
  <p>The primary action for my service (that is hosted in Kubernetes and uses Istio) is to enqueue a message to a RabbitMQ server.  This takes a bit of time, but is not CPU nor memory intensive.</p>
  <p>When I load test my service (using jMeter), neither the CPU nor memory get high enough to trigger my Horizontal Pod Autoscaling.</p>
  <p>But, the throughput is way too low for what this service should be able to do.</p>
  <p>My guess is that istio is throttling the calls to my service somehow, but I cannot seem to find what is causing it.</p>
  <h2>What I have tried</h2>
  <hr />
  <p><strong>Destination Rule</strong></p>
  <p>I made a destination rule like this:</p>
  <pre><code>apiVersion: networking.istio.io/v1alpha3
  kind: DestinationRule
  metadata:
    name: my-testing-rule
    namespace: my-testing-namespace
  spec:
    host: my-service-testing-host.svc.cluster.local
    trafficPolicy:
      connectionPool:
        http:
          http2MaxRequests: 5000 
  </code></pre>
  <p>I did this in hopes of upping the max connections, but it seemed to have no effect when I re-run the load test. (Note: the name, namespace and host in the example above have been changed from the actual values.)</p>
  <hr />
  <p><strong>No Istio</strong></p>
  <p>I tried removing Istio from the picture, and it overloaded my cluster when I fired up the load test.  I could not longer send kubectl commands and the cluster management tools would not respond.  (Apparently some throttling is good.)</p>
  <p>But this is what leads me to think that Istio is throttling things.</p>
  <h2>Question</h2>
  <p><strong>What configurations in the default setup of Istio will throttle traffic?</strong></p>
  
Tags: <kubernetes><istio>
Title: In Istio, what controls throughput to a service?
UserDisplayName: Vaccano
ThreadText: # In Istio, what controls throughput to a service?
  
  (Asked by: username_0 on 2023-06-26)
  
  I have a load testing application (jMeter) that is calling my service that is hosted in Kubernetes.
  
  ## Summary
  
  I have a service hosted in Kubernetes that seems to have limits placed on how many calls it can take. This service uses Istio (both with a Side Car and using the Istio Ingress Gateway).
  But I cannot seem to figure out what is limiting the calls to it.
  
  ## Details
  
  The primary action for my service (that is hosted in Kubernetes and uses Istio) is to enqueue a message to a RabbitMQ server. This takes a bit of time, but is not CPU nor memory intensive.
  
  When I load test my service (using jMeter), neither the CPU nor memory get high enough to trigger my Horizontal Pod Autoscaling.
  
  But, the throughput is way too low for what this service should be able to do.
  
  My guess is that istio is throttling the calls to my service somehow, but I cannot seem to find what is causing it.
  
  ## What I have tried
  
  ---
  
  **Destination Rule**
  
  I made a destination rule like this:
  
  ```
  apiVersion: networking.istio.io/v1alpha3
  kind: DestinationRule
  metadata:
    name: my-testing-rule
    namespace: my-testing-namespace
  spec:
    host: my-service-testing-host.svc.cluster.local
    trafficPolicy:
      connectionPool:
        http:
          http2MaxRequests: 5000 
  
  ```
  
  I did this in hopes of upping the max connections, but it seemed to have no effect when I re-run the load test. (Note: the name, namespace and host in the example above have been changed from the actual values.)
  
  ---
  
  **No Istio**
  
  I tried removing Istio from the picture, and it overloaded my cluster when I fired up the load test. I could not longer send kubectl commands and the cluster management tools would not respond. (Apparently some throttling is good.)
  
  But this is what leads me to think that Istio is throttling things.
  
  ## Question
  
  **What configurations in the default setup of Istio will throttle traffic?**
  
  **Tags:** kubernetes, istio
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-07-03. Score: 1
  
  You can use Envoy Filter to apply limit rate. In the below example, you apply a limit rate to a workload based on workloadSelector labels.
  The main example values are that you have a bucket of 1000 tokens. Each connection takes "one token" and in the bucket we "return" 10 tokens per 5 second. The limit rate is enabled and applied to all connection towards selected workload through `local_rate_limit_enabled` and  `local_rate_limit_enforced`.
  
  References that helped me a lot to applied the limit rate to our project was:
  
  [Limit Rate Example](https://learncloudnative.com/blog/2022-09-08-ratelimit-istio) and [Documentation](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/local_rate_limit_filter#config-http-filters-local-rate-limit)
  
  ```
  
      apiVersion: networking.istio.io/v1alpha3
  kind: EnvoyFilter
  metadata:
    name: my-limit-rater
    namespace: mynamespace
  spec:
    workloadSelector:
      labels:
        app: "myapp"
        component: "mycomponent"
        instance: "test"
        version: "1.0.0"
    configPatches:
      - applyTo: HTTP_FILTER
        match:
          context: SIDECAR_INBOUND
          listener:
            filterChain:
              filter:
                name: 'envoy.filters.network.http_connection_manager'
        patch:
          operation: INSERT_BEFORE
          value:
            name: envoy.filters.http.local_ratelimit
            typed_config:
              '@type': type.googleapis.com/udpa.type.v1.TypedStruct
              type_url: type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
              value:
                stat_prefix: http_local_rate_limiter
                enable_x_ratelimit_headers: DRAFT_VERSION_03
                token_bucket:
                  max_tokens: 1000
                  tokens_per_fill: 10
                  fill_interval: 5
                filter_enabled:
                  runtime_key: local_rate_limit_enabled
                  default_value:
                    numerator: 100
                    denominator: HUNDRED
                filter_enforced:
                  runtime_key: local_rate_limit_enforced
                  default_value:
                    numerator: 100
                    denominator: HUNDRED
  
  ```
  
  
  ---
  
  