Reliability
Rate limits and size limits
Dagon Inference limits work in flight rather than requests per minute: a key with credit can send as fast as its slots free up. Over a limit you get 429 or 503 with Retry-After.
Concurrency
| Limit | Default | Over it |
|---|---|---|
| Requests in flight per key | 8 | 429 too_many_requests |
| Requests in flight per account (a root and all its children) | 32 | 429 too_many_requests |
| Request bodies read at once per account | 2 | 429 too_many_requests |
| Request bodies without credit (no key, or a key with nothing available) read at once per client address | 4 | 429 too_many_requests |
| Requests in flight across the gateway | 64 | 503 service_busy |
The numbers are the gateway's defaults. A slot is taken once the body is read and parsed, and only by a key with credit available.
Hourly limits
| What | Limit | Scope |
|---|---|---|
POST /v1/keys | 10 an hour by default | client address |
POST /v1/credits/checkout | 10 an hour, 5 unpaid open | key |
POST /v1/credits/reconcile | 30 an hour | key |
POST /v1/messages/count_tokens | 600 an hour, 2 in flight | account |
| Child keys created | 30 an hour, 100 live | root |
| Transfers to and from children | 600 an hour | root |
| Passkey options and registration / login | 120 / 60 an hour | client address |
| x402 settlements | see x402 limits | wallet and address |
IPv6 clients are keyed by /64 (account creation and passkey limits by /48).
Body and output size
| Limit | Value |
|---|---|
| Request body with credit | 16 MiB by default |
| Request body without credit (keyless or empty key) | 256 KiB by default |
/v1/keys, /v1/credits/* bodies | 4 KB |
| Default output cap | 16,384 tokens |
| Largest output cap | 65,536 tokens, or the model's limit |
A body over the limit is refused with 413 request_too_large while it is being read.
Handling 429 and 503
import time
import openai
def create_with_retry(client, **kw):
for attempt in range(5):
try:
return client.chat.completions.create(**kw)
except (openai.RateLimitError, openai.InternalServerError) as e:
wait = float(e.response.headers.get("retry-after", 2 ** attempt))
time.sleep(wait)
raise RuntimeError("still busy")async function createWithRetry(client, body) {
for (let attempt = 0; attempt < 5; attempt++) {
try {
return await client.chat.completions.create(body);
} catch (e) {
if (e.status !== 429 && e.status !== 503) throw e;
const wait = Number(e.headers?.get?.("retry-after") ?? 2 ** attempt);
await new Promise((r) => setTimeout(r, wait * 1000));
}
}
throw new Error("still busy");
}The OpenAI SDKs also retry 429 and 5xx on their own by default.
Last updated