Skip to main content

IP, token, and service-level rate limiting with Upstash Redis or in-memory backends

Last updated November 21, 2026

Rate Limiting

Flaggr's control plane (https://flaggr.dev) applies rate limiting at three levels to protect the evaluation pipeline and ensure fair usage. Requests that the api.flaggr.dev edge sends on to flaggr.dev (bulk OFREP, the playground, /api/sdk-config, SDK telemetry, outcomes and exposure batches) count against the caller's own address: the edge passes it on.

Rate Limits

LevelLimitWindowScope
IP1,000 requests60 secondsPer client IP address (an IPv6 address counts by its /64)
Token5,000 requests60 secondsPer API token
Service10,000 requests60 secondsPer service ID

POST /api/flags/evaluate checks all three; POST /api/flags/evaluate/batch checks the IP and token limits. A request must pass every check that applies.

Response Headers

Responses from these endpoints carry one set of headers per limit that was checked:

HeaderDescriptionExample
X-RateLimit-Limit-IPMaximum requests per window for the client's IP1000
X-RateLimit-Remaining-IPRequests remaining in the window842
X-RateLimit-Reset-IPUnix time (seconds) when the window resets1721474460

The token limit uses X-RateLimit-Limit-Token, X-RateLimit-Remaining-Token and X-RateLimit-Reset-Token, and the service limit X-RateLimit-Limit-Service, X-RateLimit-Remaining-Service and X-RateLimit-Reset-Service.

Rate Limited Response

When a limit is exceeded, you receive:

HTTP/1.1 429 Too Many Requests
X-RateLimit-Limit-IP: 1000
X-RateLimit-Remaining-IP: 0
X-RateLimit-Reset-IP: 1721474460
Retry-After: 45

{
  "error": "Too Many Requests",
  "message": "IP rate limit exceeded. Please slow down.",
  "retryAfter": 45
}

The message names the limit: IP rate limit exceeded, API token rate limit exceeded or Service rate limit exceeded.

How Limits Stack

The evaluation endpoint checks the limits together:

  1. IP check: always applied, using the client's IP address
  2. Token check: applied when an Authorization: Bearer header is present
  3. Service check: applied to POST /api/flags/evaluate, using the serviceId in the request body

If any check fails, the request is rejected with 429. When more than one fails, the response reports the first in the order above.

OFREP evaluation

POST /api/ofrep/v1/evaluate/flags and /api/ofrep/v1/evaluate/flags/{key} on flaggr.dev count a request without a token against its IP. A request with a token counts against the token only, once the token is accepted, so callers that share an address keep their own allowance. A request whose token is refused, or that gets 400 before its token is checked, counts against the IP too. Once an address has spent its IP allowance, every request from it gets 429 (IP rate limit exceeded) until the window resets, before its token is checked, whatever token it sends.

Backend

UPSTASH_REDIS_REST_URL=https://your-upstash.upstash.io
UPSTASH_REDIS_REST_TOKEN=your-token

When Upstash is configured, limits use a sliding one-minute window shared across all serverless instances. Once an instance has refused an address or token, it keeps refusing it until the current one-minute window ends. This is the recommended configuration for production.

In-Memory (fallback)

When no Upstash configuration is found, rate limiting falls back to an in-memory store with a fixed one-minute window. This works for single-instance deployments and local development but does not share state across serverless instances.

Handling Rate Limits in Your Application

Exponential Backoff

async function evaluateWithRetry(flagKey: string, maxRetries = 3) {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    const response = await fetch('https://flaggr.dev/api/flags/evaluate', {
      method: 'POST',
      headers: {
        Authorization: 'Bearer fgr_your_token',
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({ flagKey, serviceId: 'web-app', environment: 'production', defaultValue: false }),
    })
 
    if (response.status !== 429) return response.json()
 
    const retryAfter = parseInt(response.headers.get('Retry-After') || '1')
    const delay = retryAfter * 1000 * Math.pow(2, attempt)
    await new Promise(r => setTimeout(r, delay))
  }
 
  throw new Error('Rate limited after max retries')
}

Use SDK Caching

The SDK caches evaluations locally. With caching enabled, you'll make far fewer API calls:

const client = createFlaggr({
  serviceId: 'web-app',
  cacheTtl: 10000, // Cache for 10 seconds
})

Use Bulk Evaluation

Instead of evaluating flags one at a time, use bulk evaluation to fetch multiple flags in a single request:

POST /api/flags/evaluate/batch
 
{
  "serviceId": "web-app",
  "environment": "production",
  "flags": [
    { "key": "checkout-v2", "defaultValue": false },
    { "key": "dark-mode", "defaultValue": false },
    { "key": "new-pricing", "defaultValue": false }
  ],
  "context": { "targetingKey": "user-123" }
}

Use Streaming

In stream mode the SDK receives the flag configurations over Server-Sent Events and evaluates locally, so evaluations stop making requests:

const client = createFlaggr({
  serviceId: 'web-app',
  apiKey: 'fgr_your_token',
  environment: 'production',
  updateMode: 'stream',
})

Monitoring

Rejections are counted in Flaggr's Prometheus metrics, for whoever operates the deployment:

# Rejections per second, by limit (ip, token or service)
sum by (type) (rate(flaggr_rate_limit_rejections_total[5m]))
 
# All 429 responses
sum(rate(flaggr_http_request_duration_seconds_count{status="429"}[5m]))

Check the Alerting page for built-in alerts that fire when error rates (including 429s) exceed thresholds.