Skip to content

Conversation

@fisherdarling
Copy link
Collaborator

@fisherdarling fisherdarling commented Dec 31, 2025

This PR allows users to rate-limit individual log invocations and adds a GCRA-based RateLimiter.

The RateLimiter supports a maximum rate along with a given burst. If the rate is 1.0/s and the burst is 3, then the limiter will allow up a burst of up to 3 requests before limiting. After being limited, a second must past before another request will be allowed. RateLimiters are thread-safe and placed in static contexts. They are cheap to create, 32 bytes or 16 bytes with a shared config, and cheap to evaluate. Checking a RateLimiter is a timestamp read + single fetch_add CAS loop. We also introduce a ratelimit! macro which can be used to easily rate-limit a given block of code.

Logging macro invocations now support a new prefix, ratelimit(burst=<expr>, rate=<expr>) which will rate-limit the log line with the given config.

For example, to log an error once per minute with a burst of 2:

error!(ratelimit(rate=1.0/60.0, burst=2), "a UDP send failed"; <tags>)

Rate-limited logs do not count against the global ratelimit.

Open Questions/Notes

  • I think the API needs some extension/more thought. Perhaps the RateLimiter uses a Cow<'a, RateLimiterConfig>. Then I think we can provide a RateLimiter::with_config(rate, burst) -> RateLimiter<'static> API. Though I'm pretty sure this increases the cost of a single RateLimiter (likely fine).
  • I've implemented my own GCRA RateLimiter here, before remembering we use the governor library for global log rate limits. governor is a nice library, we should explore making RateLimiter::direct const-compatible. On the other hand, we don't need all of governor's features and it's nice to replace a dependency with a single file.
  • Should we export ratelimit! in a more public sense? Rather than putting it behind a "private" mod or something a bit more hidden?

Future Additions

I'd like foundations to provide per-level rate limits defined in the settings. Then log invocations will automatically rate limit. Also, we could extend the logging syntax to take in a static limiter shared across multiple fns, perhaps allowing users to write something like ratelimit(limiter=<ident>).

Goal is a few-ns overhead RateLimiter for future use in log ratelimiting.
This adds the ability to rate-limit individual log invocations. It's implemented
with a prefix which can be added to any of the various logs.

E.g. to allow for one log every second, with a burst of two:
```
error!(ratelimit(rate=1.0, burst=2), "something bad happened!");
```
Previously the `iters` was divided amongst threads, meaning when the
number of threads increased, the amount of work per-thread decreased.
This had the fun effect of causing our high contention benchmark to
report ~3ns per iter...

Now every thread does `iters` amount of work and we see expected (?)
performance under contention. When we're below the number of CPUs on
my device, I see ~30ns per-iter which is what we get for the uncontended
benchmark. When we use 16 threads (more than the number of CPUs), the
threads start competing for CPU time and we jump to ~45ns per-iter.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant