API Reference

Rate Limits

Understand how rate limits and quota work on TokSpan, and how to handle 429 responses gracefully in your applications.

How Limits Work

TokSpan limits request volume and account quota so that no single customer can starve the shared infrastructure. Three kinds of limits apply:

  • Per-account rate limits — The maximum number of requests your account can make within a time window (requests/min and tokens/min, configurable per plan/group).
  • Upstream provider limits — Each upstream model provider enforces its own throughput ceilings. If a provider is saturated, requests may queue or return 429.
  • Account balance / quota — Every request consumes a small amount of your balance as it uses tokens. When your balance reaches zero, requests are rejected.

Handling 429 Responses

A 429 Too Many Requests response means you hit a rate limit or exhausted your quota. The response may include a Retry-After header telling you how many seconds to wait. Recommended handling:

  1. Check the Retry-After header, if present, and wait that long before retrying.
  2. Otherwise, retry with exponential backoff and jitter (e.g. 1s → 2s → 4s, with a small random offset) so retries don't pile up in a thundering herd.
  3. If you consistently hit limits in production, contact us to raise your account's rate limit.

Insufficient Balance / Quota

When your account or API-key balance runs out, requests are rejected. The error response uses the new_api_error type with a message mentioning quota or balance (e.g. insufficient_user_quota). This is not a rate limit — retrying will not help until you add credits.

Quota errors are not retryable. If the error message mentions quota or balance, top up in the Dashboard instead of retrying. Retrying a quota error just wastes time — the request cannot succeed without more balance.

Requesting Higher Limits

Need higher rate limits for a production workload? Contact support with your account email and expected request volume, and we'll adjust your limits or set up an enterprise plan.