Enabling Rate Limits

This task shows you how to use Istio to dynamically limit the traffic to a service.

Before you begin

  • Setup Istio by following the instructions in the Installation guide.

  • Deploy the BookInfo sample application.

  • Initialize the application version routing to direct reviews service requests from test user “jason” to version v2 and requests from any other user to v3.

    istioctl create -f samples/apps/bookinfo/route-rule-reviews-test-v2.yaml
istioctl create -f samples/apps/bookinfo/route-rule-reviews-v3.yaml

    Note: if you have conflicting rule that you set in previous tasks, use istioctl replace instead of istioctl create.

Rate limits

Istio enables users to rate limit traffic to a service.

Consider ratings as an external paid service like Rotten Tomatoes® with 1qps free quota. Using Istio we can ensure that 1qps is not breached.

  1. Point your browser at the BookInfo productpage (http://$GATEWAY_URL/productpage).

    If you log in as user “jason”, you should see black ratings stars with each review, indicating that the ratings service is being called by the “v2” version of the reviews service.

    If you log in as any other user (or logout) you should see red ratings stars with each review, indicating that the ratings service is being called by the “v3” version of the reviews service.

  2. Configure mixer with the rate limit.

    Save this as ratelimit.yaml:

    rules:
- aspects:
  - kind: quotas
    params:
      quotas:
      - descriptorName: RequestCount
        maxAmount: 1
        expiration: 1s

    and then run the following command:

    istioctl mixer rule create global ratings.default.svc.cluster.local -f ratelimit.yaml

    istioctl sets configuration for subject=ratings.default.svc.cluster.local

  3. Generate load on the productpage with the following command:

    while true; do curl -s -o /dev/null http://$GATEWAY_URL/productpage; done

  4. Refresh the productpage in your browser.

    While the load generator is running (i.e., generating more than 1 req/s), the traffic generated by your browser will be rate limited. Notice that if you log in as user “jason” or any other user, the reviews service is unable to access the ratings service, so you stop seeing stars, red or black.

Conditional rate limits

In the previous example we applied a rate limit to the ratings service without regard to any other attributes. It is possible to conditionally apply rate limits based on attributes like the source of the traffic.

The following configuration applies a 1qps rate limit only to version v3 of reviews.

  1. Configure mixer with the conditional rate limit.

    Save this as ratelimit-conditional.yaml:

    rules:
- selector: source.labels["app"]=="reviews" && source.labels["version"] == "v3"  
  aspects:
  - kind: quotas
    params:
      quotas:
      - descriptorName: RequestCount
        maxAmount: 1
        expiration: 1s

    and then run the following command:

    istioctl mixer rule create global ratings.default.svc.cluster.local -f ratelimit-conditional.yaml

    Notice the rule is the same as the previous example, only this one uses a selector to apply the ratelimit only for requests from reviews:v3.

  2. Generate load on the productpage with the following command:

    while true; do curl -s -o /dev/null http://$GATEWAY_URL/productpage; done

  3. Refresh the productpage in your browser.

    As in the previous example, while the load generator is running (i.e., generating more than 1 req/s), the traffic generated by your browser will be rate limited, but this time only if the request is from reviews:v3. Notice that this time if you log in as user “jason” (the reviews:v2 user) you will continue to see the black ratings stars. Only the other users will stop seeing the red ratings stars while the load generator is running.

Understanding rate limits

In the preceding examples we saw how Mixer applies rate limits to requests that match certain conditions.

Every distinct rate limit configuration represents a counter. If the number of requests in the last expiration duration exceed maxAmount, Mixer returns a RESOURCE_EXHAUSTED message to the proxy. The proxy in turn returns status HTTP 429 to the caller.

Multiple rate limits may apply to the same request.

Mixer MemQuota adapter uses a sliding window of sub second resolution to enforce rate limits.

Consider the following example

descriptorName: RequestCount
maxAmount: 5000
expiration: 5s
labels:
  label1: target.service

This defines a set of counters with a limit of 5000 per every 5 seconds. Individual counters within the set are identified by unique keys. A key is formed on the request path by using all parameters of the configuration. Here we introduce the notion of labels that enable creation of more granular counter keys. When a request arrives at Mixer with target.service=ratings it forms the following counter key.

$aspect_id;RequestCount;maxAmount=5000;expiration=5s;label1=ratings

Using target.service in the counter key enables independent rate limits for every service. In absence of target.service as part of the key, the same counter location is used by all services resulting in combined rate limit of 5000 requests per 5 seconds

Mixer supports an arbitrary number of labels by defining QuotaDescriptors.

name: RequestCount
rate_limit: true
labels:
   label1: 1 # STRING

Here we define RequestCount quota descriptor that takes 1 string label. We recommend using meaningful label names even though label names are arbitrary.

name: RequestCount_byService_byUser
rate_limit: true
labels:
   service: 1 # STRING
   user: 1 # STRING

Mixer expects user,service labels when the RequestCount_byService_byUser descriptor is used and produces the following config validation error if any labels are missing.

* quotas: aspect validation failed: 1 error occurred:
* quotas[RequestCount_byService_byUser].labels: wrong dimensions: descriptor expects 2 labels, found 0 labels

Cleanup

  • Remove the mixer configuration rule:

    istioctl mixer rule create global ratings.default.svc.cluster.local -f samples/apps/bookinfo/mixer-rule-empty-rule.yaml

    Note: removing a rule by setting an empty rule list is a temporary workaround because istioctl delete does not yet support mixer rules.

  • Remove the application routing rules:

    istioctl delete -f samples/apps/bookinfo/route-rule-reviews-test-v2.yaml
istioctl delete -f samples/apps/bookinfo/route-rule-reviews-v3.yaml

What’s next