RateLimiter

The Rate Limiter is a proactive resilience mechanism that can be used to limit the number of requests that can be made to a system component, thereby controlling the consumption of resources and protecting the system from overloading. A rate limiter is initialized with a configuration that, through pre-configured policies, defines its behaviour.

In this implementation, the rate limiter uses a counting semaphore synchronization primitive to control the number of permits available for requests and an internal queue to store the excess requests that are waiting for permits to be available (if configured). Since a rate limiter can be used in distributed architectures, the semaphore state can be stored in a shared data store, such as a database, by implementing the SemaphoreState interface. The rate limiter represents a resource that must be closed when it is no longer needed, as it may hold resources that need to be released (e.g., semaphore state if stored externally).

Note: How long a request holds n permits is determined by the duration of the suspending function that the rate limiter decorates, therefore is not controlled by the rate limiter itself. It is the responsibility of the caller to ensure proper timeout handling to avoid a request holding permits indefinitely.

Parameters

See also