everyais
← All docs

Streaming

Chat streaming

Set up the environment variables from the quickstart and enable stream: true. Use stream_options.include_usage: true to receive the final usage chunk.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["EVERYAIS_API_KEY"],
    base_url="https://api.everyais.com/v1",
    timeout=240.0,
)
stream = client.chat.completions.create(
    model=os.environ["EVERYAIS_MODEL"],
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
    stream_options={"include_usage": True},
)
try:
    for chunk in stream:
        if chunk.choices:
            print(chunk.choices[0].delta.content or "", end="", flush=True)
        if chunk.usage:
            print("\nUsage:", chunk.usage.total_tokens)
finally:
    stream.close()

Reading SSE events

  • Chat sends data: {...} frames and ends with data: [DONE].
  • The final usage chunk can have choices: []. Do not access choices[0] without checking.
  • Comments such as : ok are connection heartbeats, not JSON.
  • Network chunks are not SSE event boundaries. Custom consumers must buffer through a blank line and decode UTF-8 incrementally.

Messages and Responses

APICompletion and usage
/v1/messagesAnthropic events including message_start, content_block_delta, message_delta, and message_stop
/v1/responsesResponses events including response.output_text.delta and response.completed

Use the corresponding SDK's stream consumer. Do not apply Chat's [DONE] handling to every format. Responses with background: true return a job JSON object instead of a stream.

Errors and disconnections

After HTTP 200 starts a stream, provider errors can still arrive inside events. Treat error events, exceptions, or disconnection without a terminal event as failures. Close the stream when cancelling and check generated output and billing before retrying. Idempotency-Key cannot replay a completed SSE response.

Web-search citations arrive near the end. See web search for final usage and citation handling.