API

# Streaming

Set `"stream": true` on any of the three endpoints. Events are the provider's own server-sent events, passed through as they arrive.

**On this page**

- [Example](https://inference.dagon.so/docs/streaming#example)
- [How streams are metered](https://inference.dagon.so/docs/streaming#how-streams-are-metered)
- [Disconnects and broken streams](https://inference.dagon.so/docs/streaming#disconnects-and-broken-streams)

## Example

**curl**

```curl
curl -N https://inference.dagon.so/v1/chat/completions \
  -H "Authorization: Bearer $DAGON_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "abliterated-model-large-flash", "stream": true, "max_tokens": 512,
       "messages": [{"role": "user", "content": "Write a haiku about ledgers."}]}'
```

**Python**

```python
import os
from openai import OpenAI

client = OpenAI(base_url="https://inference.dagon.so/v1", api_key=os.environ["DAGON_KEY"])

stream = client.chat.completions.create(
    model="abliterated-model-large-flash",
    messages=[{"role": "user", "content": "Write a haiku about ledgers."}],
    max_tokens=512,
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
```

**TypeScript**

```ts
import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://inference.dagon.so/v1", apiKey: process.env.DAGON_KEY });

const stream = await client.chat.completions.create({
  model: "abliterated-model-large-flash",
  messages: [{ role: "user", content: "Write a haiku about ledgers." }],
  max_tokens: 512,
  stream: true,
});
for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
```

## How streams are metered

- Chat streams are sent upstream with `stream_options.include_usage` so the provider reports usage. If you did not ask for it, the extra usage chunk is removed before it reaches you.
- The receipt is signed and stored before the stream closes, so by the time you see the end it exists at `GET /v1/receipts/:id`, using the id from the `x-gateway-receipt-id` header sent with the first bytes. Streams carry no `x-gateway-receipt` header.
- Usage is taken from the final frame: chat's usage chunk, `response.completed` / `incomplete` / `failed`, or Anthropic `message_delta` with a `stop_reason`.

## Disconnects and broken streams

If you stop reading, the gateway keeps reading (without forwarding) what the provider already produced for at most 2 seconds, then cuts it off. You are charged upstream usage only if the final frame arrived; otherwise the prompt bound plus the output received, marked `usage_source: "estimated"`. That estimate can be higher than the real cost.

An error frame from the provider before any output costs nothing (a zero receipt is still signed). A stream that goes idle too long or runs past the gateway's time or size bounds is cut off and billed like a broken stream; it ends in an error.

Cap long streams

Set `max_tokens` to what you need: the hold covers the whole cap, and a smaller hold leaves more balance free for parallel requests.

Last updated 2026-11-07

[Previous **Anthropic Messages**](https://inference.dagon.so/docs/messages) [Next **Receipts**](https://inference.dagon.so/docs/receipts)