API Reference

Streaming

Set stream: true to receive the completion as a series of server-sent events instead of waiting for the full response. Any OpenAI-compatible SDK's streaming support works unchanged.

Enabling streaming

Add "stream": true to the request body. The response's Content-Type becomes text/event-stream and the connection stays open until the stream ends.

bash

curl https://api.aniron.ai/v1/chat/completions \
  -H "Authorization: Bearer $ANIRON_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "aniron/llama-3.1-70b",
    "messages": [{ "role": "user", "content": "Explain TCP." }],
    "stream": true
  }'

Chunk format

Each event is a line prefixed with data: , followed by a JSON chunk with an incremental delta instead of a full message. The stream ends with a final chunk carrying finish_reason, then a literal data: [DONE] sentinel.

text/event-stream

data: {"id":"chatcmpl-8f3a1c2b9d4e","object":"chat.completion.chunk","created":1757500000,"model":"aniron/llama-3.1-70b","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-8f3a1c2b9d4e","object":"chat.completion.chunk","created":1757500000,"model":"aniron/llama-3.1-70b","choices":[{"index":0,"delta":{"content":"TCP"},"finish_reason":null}]}

data: {"id":"chatcmpl-8f3a1c2b9d4e","object":"chat.completion.chunk","created":1757500000,"model":"aniron/llama-3.1-70b","choices":[{"index":0,"delta":{"content":" is"},"finish_reason":null}]}

data: {"id":"chatcmpl-8f3a1c2b9d4e","object":"chat.completion.chunk","created":1757500000,"model":"aniron/llama-3.1-70b","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Parsing chunks

Most OpenAI-compatible SDKs expose the stream as an async iterator, so you rarely need to parse SSE frames by hand:

JavaScript

const stream = await client.chat.completions.create({
  model: 'aniron/llama-3.1-70b',
  messages: [{ role: 'user', content: 'Explain TCP.' }],
  stream: true,
});

for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta?.content ?? '';
  process.stdout.write(delta);
}

Python

stream = client.chat.completions.create(
    model="aniron/llama-3.1-70b",
    messages=[{"role": "user", "content": "Explain TCP."}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta.content or ""
    print(delta, end="", flush=True)

Errors mid-stream

If an error occurs after the stream has started (for example the model errors out partway through generation), Aniron sends an error event and closes the connection rather than a JSON error body. Fallbacks only apply before the first token is emitted — once content has started streaming from a model, an error is not retried mid-response. See Error Handling.

Related resources

API

Wire up streaming in your app

Any OpenAI streaming client works against Aniron with no code changes beyond the base URL.

Get API key Back to docs