FixedWindowSemaphoreBasedRateLimiter
A counting suspendable Semaphore based implementation of a rate limiter that uses the fixed window counter-algorithm to control the number of permits available for requests. Uses fixed time windows (e.g., 1 second, 1 minute) to enforce the rate limit. At the beginning of each time window, the counter is reset, and requests are counted within that window. This implementation behaviour is dependent on the configuration provided (e.g., total permits, queue length, etc.).
Considerations
It's important to note that the fixed window counter algorithm can lead to:
Bursts of requests around the boundary time of the fixed window, may result in strained resources as the window counter is reset in the middle of the traffic burst.
A stampeding effect, where previously rejected requests are retried simultaneously when the time window resets, causing spikes in traffic and overload the system, especially when dealing with a large number of clients.
Parameters
The configuration for the rate limiter mechanism.
The state of the semaphore which includes an internal in-memory queue to store the excess requests.
See also
Throws
if the coroutine is cancelled while waiting for the permits to be available.
if the request is rejected due to the queue being full or the acquisition timeout being exceeded.
Properties
Functions
Calculates the duration after which the request can be retried. The retry duration depends on the rate limiting algorithm, the current state of the semaphore and the number of permits requested. For example, a token bucket algorithm may return a retry duration based on the time left until the requested permits are available (which may not happen in the next replenishment period).
Creates a RateLimiterRejectedException with the appropriate retry duration.
Deals with the caught exception while maintaining the integrity of the semaphore state.
Determines if the request should be rate limited based on the current state of the semaphore.
Depending on the rate limiting algorithm, this function is responsible for replenishing the semaphore state. For example, a fixed window counter algorithm may reset the permits to zero at the end of the window. And a token bucket algorithm may refill the bucket with tokens at a constant rate.
Updates the number of permits in the semaphore state.